为什么我的 typescript 程序随机停止运行？

Question

I wrote a very simple typescript program, which does the following:

1. Transform users.csv into an array
2. For each element/user issue an API call to create that user on a 3rd party platform
3. Print any errors

The excel file has >160,000 rows and there is no way to create them all in one API call, so I wrote this program to run in the background of my computer for ~>20 hours.

The first time I ran this, the code stopped mid for loop without an exception or anything. So, I deleted the user rows from the csv file that were already uploaded and re-ran the code. Unfortunately, this kept happening .

Interestingly, the code has stopped at non-deterministic iterations, one time it was at i=812 , another at i=27650 , and so on.

This is the code:

const main = async () => {
  const usersFile = await fsPromises.readFile("./users.csv", { encoding: "utf-8" });
  const usersArr = makeArray(usersFile);

  for (let i = 0; i < usersArr.length; i++) {
    const [ userId, email ] = usersArr[i];
    console.log(`uploading ${userId}. ${i}/${usersArr.length}`);
    try {
      await axios.post(/* create user */);
      await sleep(150);
    } catch (err) {
      console.error(`Error uploading ${userId} -`, err.message);
    }
  }
};

main();

I should mention that exceptions are within the for-loop because many rows will fail to upload with a 400 error code. As such, I've preferred to have the code run non-stop and print any errors onto a file, so that I could later re-run it for the users that failed to upload. Otherwise I would have to check whether it halted because of an error every 10 minutes.

Why does this happen? and What can I do?

I run after compiling as: node build/index.js 2>>errors.txt

EDIT: There is no code after main() and no code outside the try... catch block within the loop. errors.txt only contains 400 errors. Even if it contained another run-time exception, it seems to me that this wouldn't/shouldn't halt execution, because it would execute catch and move on to the next iteration.

Answer 1

I think this may have been related to this post. The file I was reading was extremely large as noted, and it was saved into a runtime variable. Undeterministically, the OS could have decided that the memory demanded was too high. This is probably a situation to use a Readable Stream instead of a readFile .