Can Inotify handle a large amount of directories?

Question

Has anyone used inotify to watch something like 500K - 1M directories? How's the performance on such a large amount of watches?

I was thinking to create a server that tracks file changes on some sites. The server would be written in PHP or nodejs . Before I start I'd like to know if inotify is the right tool for this task.

I can increase the max number of watches:

fs.inotify.max_user_watches=1000000

But what next? Can inotify handle it?

Answer 1

inotify itself can handle that, provided that your system can handle the workload. For example, if the directories are rarely changed then you only need sufficient RAM to maintain the state. If the directories are actively and frequently undergoing changes then you will need to make sure that the service that is consuming the notifications can keep up and that your storage device can sustain the IO of those changes.

You will definitely need to increase the various inotify system limits to accommodate that number of watches.

You may wish to consider using watchman to handle the heavy lifting and let you focus on your application logic.

You may also find What are the performance and reliability implications of watching too many files? useful.