Too Many Files Open while monitoring file changes

Question

I am developing a document-browser-based app for the iPad. I have been using SKQueue to monitor changes to the files in order to make sure their metadata remains current when the user performs actions in the document browser. The code for initiating monitoring:

    // Set up the queue
    let delegate = self
    queue = SKQueue(delegate: delegate)!
    // Remove all existing paths
    queue?.removeAllPaths()

    // Get the list of PDF URLs using a function that enumerates a folder's contents
    let pdfFiles = getFolderContents(rootFolder: myDocumentsFolder, extensionWanted: "pdf")
    for pdfFilePath in pdfFiles.filePaths {
        queue?.addPath(pdfFilePath.path)
    }
    for pdfFolderPath in pdfFiles.folderPaths {
        queue?.addPath(pdfFolderPath.path)
    }

I developed my own logic to respond to notifications from this queue, but I do not remove any items from the queue during the app's runtime.

The problem - it seems that when the number of items watched is over 200 (files and folders) the system hits a wall and the console reports error 24: Too Many Files Open. After that, no reading/writing of any file can be performed.

From what I was able to gather from searching, it seems that iOS and iPadOS do not allow more than 256 files descriptors to be accessed at the same time, and that would mean that the GCD approach to monitoring file changes would suffer from the same limitation.

Is there any way to monitor files changes that is not subject to such a limitation? Any other suggestions?

Answer 1

After a lot of research and experimentation, I can finally verify that indeed, the default maximum number allowed of open file descriptors is 256 - for MacOS, iOS and iPadOS. This can be changed easily in MacOS - see article here . However, iOS and iPadOS being much more closed in nature, there is no reliable way of changing this limit on these platforms.

Good practices are therefore:

1. Try to avoid designing an app that would call for so many file descriptors open.
2. Monitor directories, and not individual files. With most tools available for monitoring the file system, you get notifications for any file change within a monitored directory. Just implement your logic from there, by enumerating the folder and comparing its new state with a saved state. You can find good code for that in the top two answers of this SO thread .

Note: I recommend enumeration rather than other methods of getting the file system state, because other methods tend to give incompatible results between the simulator and actual devices (different treatment of symlink resolution).

3. Make sure your method of choice for monitoring the file system can be queried for the number of items being watched, and issue an alert if the user approaches a set limit. Remember that the 256 open files must also include all files used by the app, including ones in the app's bundle and ones that are in actual use. So take a nice safety margin.

In my case, my app uses a UIDocumentBrowserViewController, or in other words - Apple's own Files app, to allow users to manage their files. I have to keep my metadata up-to-date with the file system state, and I have no control over the file management habits of my users. To complicate matters, the Files app itself can be used to modify the app's file system - while my app is not active.

Therefore, I do two things:

1. I save a detailed state of the file system from the App Delegate's applicationDidEnterBackground and applicationWillTerminate methods into a json file in Application Support, and on the app's launch I compare it to a fresh enumeration of the file system - and alert the user for any mismatch, suggesting to use the app's own file browser next time.
2. I created my own swift package, named SFSMonitor , for monitoring the file system. It is based on the wonderfully convenient SKQueue (which I highly recommend), but instead of monitoring kevent, it uses Dispatch Sources - a more modern approach, which Apple advocates. It is similar to Apple's own Directory Monitor (reference to which you can find here ), but instead of monitoring one directory, it allows you to create and manage a whole queue of them. This class allows you to set a maximum number of monitored file descriptors, and be notified when that limit is reached.