Windows. Service multiple instances processing files from same single directory duplication

Question

I have .net windows service which gets list of image files from a folder and do some conversion and sent the converted files to another directory. I want achive more throughput by adding another instance of serVice watching same folder. I want 2 instances process files independently without any duplicate processing. What patterns can be used? Is file locking would work for this ? Don't want to use database or any other messaging platform. I Can use text files etc to create synchronization if needed.

Answer 1

1. If using .net I would consider creating multiple threads (using TPL in .net) that would be used to process the files in parallel. This way you have a single process that has control over the entire process. Hence no need to track what process (exe) is processing a file, no databases, no locking, etc..

2. However if you wish to have multiple processes processing the files, then one option of synchronizing the processing is to make use of a Mutex.

I would use this option along with Solution 1. Ie use TPL (multiple threads) in one service. And also use Mutexes. This way you have the benefit of multiple threads and multiple services. Hopefully this is what you are after.

https://msdn.microsoft.com/en-us/library/bwe34f1k(v=vs.110).aspx

Before processing any file, create a Mutex with a particular name and if ownership has been granted, then continue processing the file. If ownership hasn't been granted you can safely assume that another process or another thread (within the same application) has acquired a lock on this Mutex, meaning another process/thread is already processing the file.

Sample code:

var fileMutex = new Mutex(true, "File Name", out mutexWasCreated);
if (mutexWasCreated){
    //Some other process/thread is processing this file, so nothing to do
}
else {
    //Start processing the file
}

If one service (exe) goes down, then the threads would die, meaning the mutexes would be released and those files will be available for processing by another process.