File system watcher for Mac OS X

Question

Currently, we use QFileSystemWatcher, belonging to Qt. Due to the limited support in Mac OS X, it can only notify us two events: directory changed, or file changed.

However, the latter event (file changed) is triggered multiple times when its size is slightly larger and the writing to disk takes slightly longer time.

Our workaround is to set up a timer to check the file in 1 second. If more signals about the file come before timer expires, we reset the timer.

Is there a way to get the notification when the file is written to the disk (finished writing)? There is no need to limit to Qt, any library could do.

---

We are aware of the kqueue monitoring method, but that's too low level and we don't want to do that for each file, as we are monitoring a large portion of the file system..

Answer 1

I have the same problem in a project and finally I decide to implement a native watcher. It's pretty easy:

In the .h:

class OSXWatcher : public Watcher
{
public:

    OSXWatcher(const QString& strDirectory);
    virtual ~OSXWatcher();

    virtual bool Start();
    virtual bool Stop();

private:

    /**
     * Callback function of the OS X FSEvent API.
     */
    static void fileSystemEventCallback(ConstFSEventStreamRef streamRef, void *clientCallBackInfo, size_t numEvents, void *eventPaths, const FSEventStreamEventFlags eventFlags[], const FSEventStreamEventId eventIds[]);

    FSEventStreamRef stream;
};

The .cpp:

bool OSXWatcher::Start()
{
    CFStringRef pathToWatchCF = CFStringCreateWithCString(NULL, this->dirToWatch.toUtf8().constData(), kCFStringEncodingUTF8);
    CFArrayRef pathsToWatch = CFArrayCreate(NULL, (const void **)&pathToWatchCF, 1, NULL);

    FSEventStreamContext context;
    context.version = 0;
    context.info = this;
    context.retain = NULL;
    context.release = NULL;
    context.copyDescription = NULL;

    stream = FSEventStreamCreate(NULL, &OSXWatcher::fileSystemEventCallback, &context, pathsToWatch, kFSEventStreamEventIdSinceNow, 3.0, kFSEventStreamCreateFlagFileEvents);
    FSEventStreamScheduleWithRunLoop(stream, CFRunLoopGetCurrent(), kCFRunLoopDefaultMode);
    FSEventStreamStart(stream);

    CFRelease(pathToWatchCF);

    // Read the folder content to protect any unprotected or pending file
    ReadFolderContent();
}

bool OSXWatcher::Stop()
{
    FSEventStreamStop(stream);
    FSEventStreamInvalidate(stream);
    FSEventStreamRelease(stream);
}

void OSXWatcher::fileSystemEventCallback(ConstFSEventStreamRef /*streamRef*/, void *clientCallBackInfo, size_t numEvents, void *eventPaths, const FSEventStreamEventFlags eventFlags[], const FSEventStreamEventId eventIds[])
{
    char **paths = (char **)eventPaths;

    for (size_t i=0; i<numEvents; i++) {
        // When a file is created we receive first a kFSEventStreamEventFlagItemCreated and second a (kFSEventStreamEventFlagItemCreated & kFSEventStreamEventFlagItemModified)
        // when the file is finally copied. Catch this second event.
        if (eventFlags[i] & kFSEventStreamEventFlagItemCreated
                && eventFlags[i] & kFSEventStreamEventFlagItemModified
                && !(eventFlags[i] & kFSEventStreamEventFlagItemIsDir)
                && !(eventFlags[i] & kFSEventStreamEventFlagItemIsSymlink)
                && !(eventFlags[i] & kFSEventStreamEventFlagItemFinderInfoMod)) {

            OSXWatcher *watcher = (OSXWatcher *)clientCallBackInfo;
            if (watcher->FileValidator(paths[i]))
                emit watcher->yourSignalHere();
        }
    }
}

Answer 2

I have the same problem but with folder. When you copy many files to the folder too many singals are emitted but I need only one. So I have the following solution:

void folderChanged(const QString& folder)
{
    m_pTimerForChanges->start();
}

folderChanged is a slot for directoryChanged() signal. And timer has another connection for timeout, so when time is out then processing should be done. Timer has 1s interval. Idea behind it is that folder should not be updated more frequent than interval I have and if it sends signals more frequently than I need then I don't need to process them immediately. Rather I restart timer every time the signal is emmited and with all of it I have the only one processing of changes. I think you can apply the same approach.

Another approach which may work for you also is to check file modification date in your processing and if its current modification date is within some epsilon(small interval) with your last modification date then you have repeating signal and should not react on it. Store this modification date and proceed.