Thread safe file access

Question

I have a program that does a large number of comparisons. It compares a specific .dat file saved on the local machine to a large number of other files that are generated on run-time. Right now I am unable to perform these comparisons using multiple threads because of many System.AccessViolationException . I'm assuming this is because multiple threads are trying to access the same local file at the same time. How can I overcome this to do these comparisons with multiple threads?

Answer 1

There are several possible reasons for your access violation:

1. Multiple threads are exclusively locking your specific .dat file
2. Your multi-threading is buggy in the regard that multiple threads try to read the same runtime generated file
3. Your multi-threading is buggy in the regard that your threads try to read the runtime generated files before they have been generated completely

The following solutions exist:

1. Read the .dat file into memory once and share that data between all threads. This also reduces the I/O load
2. Make sure every runtime generated file is compared only by one thread. This can be achieved by a thread safe queue that contains all files that need to be compared and that is shared between all threads.
3. Make sure that the runtime generated file gets known to the reading threads only after it has been created completely. This can be achieved by creating it in a different directory on the same disk and moving it to the target directory putting the file name into the queue from solution 2 only after creation is completed.

---

As Matthew Watson correctly points out, an AccessViolationException is caused by errors in unmanaged code, so it is probably not caused by multiple threads trying to access the same file.
My answer therefore assumes that you are actually getting an UnauthorizedAccessException .
If that is not the case and you are indeed getting an AccessViolationException your problem most likely lies elsewhere.