The problem
When loading from source it is common for things to suddenly become slow and require investigation. If the source extraction suddenly pulls in more rows, that would explain an increase in runtime. We cannot see this directly but need to query the Ingest storage outside of TimeXtender products. This is especially annoying for parquet.
Who is affected
As a partner consultant, this is something that comes up regularly. Either in implementations or when helping customers with issues. This should affect all users using Ingest and ODX Server
What you do today
Depending on the storage engine I either open SSMS, navigate to the database, deduce the correct table and generate a query to count rows. This can require some logic to be able to see the changes in rowcounts for incrementals. As there is no formal documentation for the way Ingest storage is implemented, I cannot know for certain my conclusions are correct. For parquet I may need to write a notebook in python/whatever language to count the rows if the file is too large or use a SQL Analytics endpoint in Fabric or run a query in Snowflake, otherwise I need a tool to open the parquet.
What would help
I would like to be able to see rowcounts in Ingest, similar to how this is available for Prepare tables or BU tables in TX Classic.