react-native start gives me “Watchman error: too many pending cache jobs”

Question

Problem

When I create a bundle - the watchman crawl initially fails with " too many pending cache jobs ", which seems to slow down the bundling process:

$ react-native start --reset-cache

Loading dependency graph...jest-haste-map: Watchman crawl failed. Retrying once with node crawler.

Usually this happens when watchman isn't running. Create an empty .watchmanconfig file in your project's root folder or initialize a git or hg repository in your project.

Error: Watchman error: too many pending cache jobs. Make sure watchman is running for this project. See https://facebook.github.io/watchman/docs/troubleshooting.html

Also, these may be related:

• I can get rid of the error by excluding node_modules from the watch config
• Without the error, and the bundler starts in 20 seconds
• With the error, the craw takes 4 seconds, but the bundler starts in 4 minutes

Background

Pre-bundle scripts

I run these before the bundler, to attempt to get more stability:

watchman watch-del-all
watchman shutdown-server
sudo sysctl -w kern.maxfiles=5242880
sudo sysctl -w kern.maxfilesperproc=524288

Watchman config file

I set my .watchconfig file to be extra permissable:

{
  "ignore_dirs": [],
  "fsevents_latency": 0.5,
  "fsevents_try_resync": true
}

Watchman log

The crawl does complete , within a few seconds according to the full log

$ tail -f /usr/local/var/run/watchman/coolman-state/log

22:08:24,193: [client=0x7ff4a9686d98:stm=0x7ff4a9b2de00:pid=0] send_error_response: too many pending cache jobs
22:08:24,340: [listener] Watchman 4.9.0 <no build info set> starting up on coolman
22:08:24,341: [listener] path /Users/coolman/project/react-native-app is on filesystem type apfs
22:08:24,342: [listener] root /Users/coolman/project/react-native-app using watcher mechanism fsevents (auto was requested)
22:08:24,344: [listener] file limit is 2560 kern.maxfilesperproc=524288
22:08:24,344: [listener] raised file limit to 524288
22:08:24,345: [listener] launchd: "sock" wasn't present in Sockets
22:08:28,805: [io 0x7f87ba000818 /Users/coolman/project/react-native-app] PERF: {"ru_nvcsw": 1460, "ru_nsignals": 0, "ru_msgrcv": 0, "ru_msgsnd": 0, "ru_inblock": 0, "ru_majflt": 0, "ru_nswap": 0, "ru_minflt": 28103, "ru_ixrss": 0, "ru_maxrss": 115109888, "system_time": 2.9349970000000001, "user_time": 1.327971, "elapsed_time": 4.4614079999999996, "pid": 29414, "ru_idrss": 0, "meta": {"root": {"watcher": "fsevents", "ticks": 2, "number": 1, "case_sensitive": false, "recrawl_count": 0, "path": "/Users/coolman/project/react-native-app"}}, "version": "4.9.0", "ru_oublock": 0, "ru_nivcsw": 2171, "start_time": 1564952904.3444469, "description": "full-crawl"}
22:08:28,805: [io 0x7f87ba000818 /Users/coolman/project/react-native-app] crawl complete

Answer 1

Solution

Adding "content_hash_max_items": 500000 to my .watchmanconfig solved it for me.

Explanation:

Sounds like you need to tune the metadata cache size(s) for your workload; that error occurs when watchman is attempting to insert more cache entries than are configured.

You can specify larger values in your .watchmanconfig file:

• content_hash_max_items is related to the content.sha1hex metadata and defaults to 131072 . A related option is content_hash_max_warm_per_settle which defaults to 1024 and which causes up to this many items to be assessed on each settle.
• symlink_target_max_items defaults to 32768 . This cache is used for readlink in the case that queries ask for the symlink target. This is less likely to be your issue.

I would suggest running find /Users/coolman/project/react-native-app | wc -l find /Users/coolman/project/react-native-app | wc -l to get an upper bound on the number of files currently in your project and use that to size these caches; if you have more files than those defaults then you should raise those cache settings accordingly.

I do not recommend changing the fsevents_latency parameter from the default; can you share more context on what led you to change it?

Github