IO用node-js编写

Question

I have a node.js (syntax is Typescript actually) application that:

• at some HTTP request handler asynchronously write in a file a daily count and
• a cron job (inside the node application done with node-cron module) that at midnight reset that file

The non-blocking event loop loop is not super clear to me ( it's not single thread under the hood if I got it right) and I'm worried about situation where the cron module is resetting the file while I'm writing to it.

Do I have to worry about it? Like with a global flag that is set while my promisified fs.writeFile is writing? Are there some more elegant way to handle it?

Thanks and sorry if it's a dumb question.

This is a skeleton of my code:

import * as fs from 'fs';
import * as path from 'path';
import { CronJob } from 'cron';
import { pfs } from './promisifiedFs';

const daily_file = '/path_to_my_file'


new CronJob('0 0 * * * *', function() {
  fs.writeFileSync(daily_file, 0, {'flag': 'w'});
}, null, true);


// somewhere called inside an HTTP GET handler
async function doBill(data) {

  const something = //....

  const currentCountRaw = await pfs.readfilePromisified(daily_file, 'utf-8');
  const currentCount = parseFloat(currentCountRaw) || 0;
  await pfs.writeFilePromisified(daily_file, currentCount + something, {'flag': 'w'});

}

Answer 1

In Node.js your code runs in a single thread. It is single threaded unless you are using native modules that do asynchronous I/O that happen to load off work to the thread pool (which isn't always the case) or if you use the cluster module.

As you are using writeFileSync in your script the main thread will block until you are done doing that and will do nothing else in parallel.

Edit: As noted in the comments there is a case that is probable to happen: The cron job runs between the read and the write. There is another case that is unlikely to happen but possible: The async write is already running and the cronjob runs. So I revert what I said because it was wrong and would recommend creating a mutex like variable or a lock file.

Answer 2

I would suggest using something like async queue (if you are only running one instance of the node application) or lockfile (if you intend to cluster or use multiple processes).

Here's some code which (I believe) will duplicate the issue I have outlined above, by writing much larger values which should increase the chance of a multiple write collision.

What my computer seems to do is take the fs.writeFileSync and... do nothing with it? I could be mistaken, but here's the code I'm using. I opted for co instead of async/await to avoid needing to transpile.

const fs = require('fs');
const co = require('co');
const crypto = require('crypto');

function wait(ms) {
  return new Promise(resolve => {
    setTimeout(resolve, ms);
  });
}

function now() {
  return new Date().toISOString();
}

function log(...args) {
  return console.log(now(), ...args);
}

const file = './output.txt';

log('generating bytes...');
const bytes = crypto.randomBytes(100000000);
log('done generating bytes');

function writeAsync() {
  return new Promise((resolve, reject) => {
    return fs.writeFile(file, bytes.toString('base64'), { 'flag': 'w' }, err => {
      if (err) {
        return reject(err);
      }
      return resolve();
    });
  });
}

function writeSync() {
  fs.writeFileSync(file, 'FINDMEFINDMEFINDME', { 'flag': 'w' });
}

function run() {
  return co(function*() {
    log('before write async');
    const promise = writeAsync().then(() => log('after write async'));
    const ms = 1;
    log(`waiting for ${ms} ms`);
    yield wait(ms);
    log('done waiting');
    log('before write sync');
    writeSync();
    log('after write sync');
    yield promise;
  });
}

run().catch(err => console.error(err));

Example output:

2016-10-03T22:52:05.032Z generating bytes...
2016-10-03T22:52:06.846Z done generating bytes
2016-10-03T22:52:06.848Z before write async
2016-10-03T22:52:06.999Z waiting for 1 ms
2016-10-03T22:52:07.001Z done waiting
2016-10-03T22:52:07.001Z before write sync
2016-10-03T22:52:07.012Z after write sync
2016-10-03T22:52:08.623Z after write async

Behavior:

It seems like the file output contains the result of writeAsync() and writeSync() has no effect on the output. I sort of expected an error to be thrown, to be honest. Maybe even writeSync() placing 'FINDMEFINDMEFINDME' in the middle of the other code, but I wasn't able to find that string in the output file.