How do I write a windows service that polls a directory for new files?

Question

I have made a window service, it is basically reading a temporary file on my machine, reading it to a database and then deleting it. I have written the code for these actions in the onStart method, hence I need to restart the service again when I need it to work, but what I actually need is that the service should automatically sense the creation of the file in my folder and then work on it.

So where should I place this particular code in my windows service?

Answer 1

Since FileSystemWatcher isn't guaranteed to notify you on all file system changes I would recommend polling it using a Timer with a set interval instead. This approach is easier on system resources too.

Internally, FSW buffers events received from the file system. If too many events occur at the same time this buffer will overflow and you will start losing events. Unless your event handler code finishes really quickly or you use some queuing mechanism, you will also start losing events. IMO, this makes FSW a less-than 100% reliable approach.

Here is an article that discusses FSW vs performance in more detail.

A note on system resource: FSW relies on registering a callback with the OS file system. I have no metrics on how much this accounts for in system resources. My suggestion of using a Timer actively polling the system requires no such resources from the file system. You could poll the file system every 30 seconds or every 5 minutes depending how fast you need the files to be picked up.

Unless you really need near-realtime behavior that is...

Answer 2

FileSystemWatcher

http://msdn.microsoft.com/en-us/library/system.io.filesystemwatcher.aspx

Answer 3

Maybe a combination approach could be useful? Combining a timer and a file system watcher.

Using a temporary extension when writing to the file and renaming when ready could also be useful to prevent reading file while writing to it.

Answer 4

Whether you use a timer or a FileSystemWatcher, both will fire events that you need to subscribe to. When a file is detected, an event is fired. This starts a new thread which executes your event handler.

Therefore, in your OnStart method you need to subscribe to the event. For this, your event handler needs to be a method that is somehow accessible from your OnStart().

For the record, I got burned by the FSW several times and don't use it any more, I find it too unreliable. I wrote my own FileWatcher class that does what Peter Lillevold suggests and uses a timer to repetedly poll the directory. Perhaps a bit crude, but it works. And you can't beat that ;-)

Answer 5

I recently created something that is almost exactly what you are describing. At first I thought I should use the FileSystemWatcher. Problem is it only notified me of things as they happened. If the service was down and had to be restarted for whatever reason, when it started up it did not fire any events regarding the files that have been created while the service was down.

So I stared writing my own poller that would check for files at startup of the service and then use the FileSystemWatcher. I then started thinking, why even bother with the FileSystemWatcher anyways and have 2 pieces of code to maintain? So I dropped it completely and just have my poller do it all since the app just needs to do what the poller did at the startup over and over.

My app is a windows service. The OnStart creates a thread that runs a function to do the polling. That function is just a while loop with an end condition triggered by the OnStop and it just polls every minute and scans the directory. If it finds something it deals with it, if not it sleeps for 1 minute. Pretty simple. The code is basic and easy to maintain as well.

Here is a sample of my OnStart:

    protected override void OnStart(string[] args)
    {
        WriteToLog("Starting...");
        bThreadRun_m = true;

        tMain_m = new Thread(new ThreadStart(RunThread));
        tMain_m.Name = "Main Thread";
        tMain_m.Start();

        WriteToLog("Start Complete");
    }