How should I deal with an occasionally corrupted file?

Question

I have an app that saves data in sessions of 15 minutes each. The saved data accumulates until it reaches a size threshold and then starts deleting the oldest data first. However, the data includes sensitive information, so when the data is written is scrubbed from the file, including the last modified date. I use a counter as part of the file name to keep track of the file order. I need to keep track of the counter through reboots and even if the data itself is deleted. As a solution, to determine the oldest file, I keep a simple counter in a 1-line text file. Every 15 minutes, I read from this file, increment the counter, and write the result back to the file:

int catalogNumber = 0;
if (File.Exists(catalogFilePath))
{
    catalogNumber = int.Parse(File.ReadAllText(catalogFilePath);
    catalogNumber++;
}

File.WriteAllText(catalogFilePath, catalogNumber.ToString());

This works almost always. However, once in a great while (maybe once a year or so), something gets messed up. I suspect it's because a user has killed the app during the short window when File.WriteAllText() is being called, but the result is that the next time through the loop the read fails. My question is how can I ensure that I never lose the counter?

Answer 1

Here is a likely culprit.

WriteAllText uses a StreamWriter internally, with a small buffer size.

The buffer is the internal array used by a stream, when the array is full it tends to flush it to disk, that's to say a stream isn't atomically written to disk.

As such, there is an edge case that only a partial file gets written on abnormal shutdown (when the WriteAllText only partially finished).

A solution to this might to use your own streams and increase the buffer beyond the size of the data, however this is far from bullet proof.

Another solution would be to write to a temporary file, then when it's finished, use File.Move to move the temp file to your actual destination file. File.Move will use the native win32 api MoveFile it will in turn be handled internally by the Operating System atomically (all or nothing).

The result will be, you will have a complete destination file or no file / no change, depending on what you are doing (and disregarding any other problem).