限制使用 nodejs 观看的文件数量

Question

I have a script written in nodejs with chokidar where I listen into a folder when a file is added.

When a file is added start a function with many loop.

Sometimes the system add 100 or more files at the same time so the event add is called 100 times and the server can crash.

Is there a way to limit watched file and the others wait until watched file are closed?

For example I decide that I can't watch more than 10 files at the same time, the other files wait until first 10 finished their thing.

Is possible?

This is my script:

var chokidar = require('chokidar');
var sys = require('sys')
var exec = require('child_process').exec;

var folder = process.argv[2];
var watcher = chokidar.watch('tmp/xml/' + folder + '/', {ignored: /^\./, persistent: true});
watcher
  .on('add', function(path) {
    var filenameArr = path.split('/');
    filename = folder + '/' + filenameArr[filenameArr.length - 1];

    function puts(error, stdout, stderr) { sys.puts(stdout) }
    exec('bin/cake manager readXmlAndSave ' + filename, puts);
})

Answer 1

const chokidar = require('chokidar');
const async = require("async");

[...]

let taskQueue;
const watcher = chokidar.watch('myPath');
watcher
  .on('add', function(path) {
    taskQueue.push({path});
})

// handle files 10 per 10 to avoid to exhaust critical resources
taskQueue = async.queue(async function(task) {
  await myAsyncHandler(task.path);
}, 10);

taskQueue.error(function(err, task) {
  handleError(err, task);
});

For my culture I'd love to see a pure vanillaJS answer using for example streams and backpressure