Bypass order by in subquery for getting recordcount of query

Question

I have a function which executes a datareader with a given commandtext and invokes a given action for every data record returned by the reader.

Since the datareader is a forward-only reader i can not get the count of rows from the datareader. For invoking a progressbar i have to know the count of the records my datareader will return.

My current attempt looks like this:

protected void RunConversion(string commandText, Action<IDataRecord> actionConversion)
{
    try
    {
        var curIndex = 0;
        var maxIndex = this.Source.ExecuteScalar<int?>($"SELECT COUNT(*) FROM ({commandText}) AS QUERY"); //Here i try to get the count of rows from the result 

        OnProgress?.Invoke(0);

        foreach (var dataRecord in this.Source.ExecuteReader(commandText))
        {
            try
            {
                actionConversion?.Invoke(dataRecord);
            }
            catch (Exception e)
            {
                OnReport?.Invoke($"Error: {e}{Environment.NewLine}");
            }
            finally
            {
                OnProgress?.Invoke((int)((double)++curIndex / maxIndex * 100f));
            }
        }
    }
    catch (Exception e)
    {
        OnReport?.Invoke($"Fatal Error: {e}{Environment.NewLine}");
        OnProgress?.Invoke(100);
    }
}

The problem is, sometimes the commandText-Parameter contains a query which includes order by... without using top the order by is not allowed in subqueries.

Another attempt would be to provide a thrid parameter which is a query that only counts the result of my current commandText-Parameter. This would be no nice solution, since i have to write two queries and they can be pretty complex.

Any idea for a simple solution?

Answer 1

You mention you can't use the count because you are using a DataReader . Are you showing a Progress Bar for the query? If so, I would suggest that instead of a progress bar, simply show a "wait" icon rather than executing a query twice and extending the time that the user has to wait just to display how long they will be waiting.

With that in mind, you have two options. The first being to do away with the progress bar and the SELECT COUNT(*) FROM (...); and simply pass the query as-is. Then when you execute it and get a result set back, you can count the records if you still need to show a progress bar for record processing. Granted, this has the downside of adding data transfer overhead to your operation if you aren't going to use the actual data, so it is up to you if it is worth that little extra debt.

Your second option would be to simply remove the ORDER BY portion of any queries by doing something like the following:

public string RemoveOrderBy(string query)
{
    return query.SubString(0, query.IndexOf("ORDER BY"));
}

This would get rid of any ORDER BY statements, but if you have any other things that may not work in a sub-query you would have to account for them in the same way as above.