SQL Server CE & C# After deleting records cannot read data in a table

Question

I have created a database file that updates readings every minute and stores in a SQL Server CE database file. However as the database gets very large, it starts to get really slow.

I decided to delete the oldest files once the database reaches a certain size as they are of no use to me. I managed to do this using the following commands:

command.Append("DELETE FROM MyTable WHERE ID IN (SELECT TOP(" + difference.ToString() + ") ID From MyTable)");

where difference.ToString() is value I used to calculate how much I want to delete.

This worked successfully as I could open the file using CompactView and also I could type in the commands in CompactView to give the same results.

Now my problem started when I tried to read the data and update it on to a graph. So my codes in another form does the following:

private void updateGraphTimer_Tick(object sender, EventArgs e)
{
    using (SqlCeConnection connection = new SqlCeConnection(connectionString))
    {
        connection.Open();

        using (SqlCeDataAdapter adapter = new SqlCeDataAdapter("SELECT * FROM MyTable", connection))
        {
            // some code that is not relevant between these two statements
            using (DataTable table = new DataTable())
            {
                StringBuilder command = new StringBuilder();
                command.Clear();
                command.Append("SELECT TOP(1) ID FROM MyTable ORDER BY ID DESC");

                using (SqlCeCommand com = new SqlCeCommand(command.ToString(), connection))
                {
                    int value = (int)com.ExecuteScalar();
                    graphPage.latestID = value;

                    if (value > graphPage.startID)
                    {
                        DataColumn xDateColumn;
                        xDateColumn = new DataColumn("XDate");
                        xDateColumn.DataType = typeof(double);
                        table.Columns.Add(xDateColumn);

                        adapter.Fill(graphPage.startID, value, table); 

The problem I have is that the table is empty, even though value from (int)com.ExecuteScalar() returns a value! If I did not perform the delete, it all works fine!

I cannot figure out what is happening! The only thing I can think of is something to do with reading and writing the sql file.

Much appreciated!

Answer 1

Better not use TOP here. As ID is incremented automatically, get the highest ID and then delete at max before that ID as the latest ID is used in the graphing part. For example, delete everything below current max ID or below max ID - 10 or so.

DELETE * from table where ID < MAX(ID);

If using TOP as you do in the delete, all values, including the TOP (last ID) is deleted. And in the graphing part you are accessing the last ID too.

Answer 2

So you have two problems:

1. your table is empty after the delete
2. your query returns a result in spite of the fact that the table is empty

Regarding the first one the problem is in your delete statement: for some reason your difference value is higher or equal to the number of rows in your table and will delete the overall table . I would look at the way you are computing it. In any case if you have an insert_date on your table I would change the delete statement like this to delete rows older than 10 days from today:

DELETE FROM MyTable WHERE ID IN (SELECT ID From MyTable where insert_date < DATEADD(day, -10, GETDATE())

The second problem is a concurrency issue related to the fact that two threads are working: it may happens that you read data (and data are available) just before data are deleted; so you are not really getting a query result from an empty table also if this is the impression.

Answer 3

I think I figured out the problem! Forgive me if you tried to explain this to me and I didn't get it but here goes! The problem is with populating the table whilst the value returned from ExecuteScalar is correct, that Id is greater than the table as the table counts from 0 rather than the subtracted amount!

So for example I deleted 1000 records from 5000, the oldest record would then be 1000, so when I fill the table the 5000 is the latest id, however I am trying to access the 5000th record in the table rather than the 4000th value which contains ID 5000!

I will try and rewrite the code take this into account, this was never a problem before as when I wasn't deleting they matched. D'OH!