Abort this node-fetch request without aborting all other requests

Question

. 。 Answers to this question are eligible for a +200 reputation bounty. stkvtflw wants to to this question. 。

The only way I found to set a timeout on a node-fetch request is to use the abort-controller :

import AbortController from 'abort-controller'
import fetch from 'node-fetch'

const fetchWithTimeout = async (url, options) => {
  const controller = new AbortController()
  const timeout = setTimeout(() => {
    controller.abort()
  }, 10000)
  const response = await fetch(url, {
    signal: controller.signal,
    ...options
  })
  clearTimeout(timeout)
  return response.text()
}

Besides the fact that this looks ugly, it also has a huge issue that I don't know how to dodge: The controller.abort() aborts ALL current requests, not just one. So, if I call fetchWithTimeout() 10 times in parallel and one of them gets aborted, then all the others get aborted too.

I've kinda solved it with even more awkward code:

import AbortController from 'abort-controller'
import fetch from 'node-fetch'

const aborted = {}
const fetchWithTimeout = async (url, options) => {
  const controller = new AbortController()
  const timeout = setTimeout(() => {
    aborted[url] = true
    controller.abort()
  }, 10000)
  try {
    const response = await fetch(url, {
      signal: controller.signal,
      ...options
    })
    clearTimeout(timeout)
    return response.text()
  } catch (e) {
    clearTimeout(timeout)
    if (aborted[url] || e?.type !== 'aborted') {
      delete aborted[url]
      throw e
    } else {
      return await fetchWithTimeout(url, options)
    }
  }
}

But this is so wrong and inefficient. What else can I do with it besides replacing node-fetch with an alternative?

Answer 1

I tried to reproduce the behaviour of controller.abort() aborting ALL current requests, but it seems to work as expected and I doubt that it's a bug in node-fetch .

I think the problem is how you handle the errors when you execute the requests in parallel (presumably) using Promise.all . As stated in the docs it rejects immediately upon any of the input promises rejecting or non-promises throwing an error, and will reject with this first rejection message / error.

You are probably looking for Promise.allSettled instead, as it resolves after all of the given promises have either fulfilled or rejected independently of each other. So it could look something like this:

(async () => {
    const parallelRequests = [];
    for (let i = 0; i < 5; i++) {
        parallelRequests.push(fetchWithTimeout('http://some-url.com'));
    }
    const result = await Promise.allSettled(parallelRequests);
    // result will be an array holding status information about each promise and the resolved value or the rejected error
})();

The reason why your second approach works with Promise.all , is that you actually catch and handle errors from the fetch-call. But be aware that in case of non-aborted error, you re-throw the error which will again cause an immediate rejection regardless of the other requests.

Answer 2

I've spent too many hours on various node fetch libraries. Just use curl. It just works.

import { exec } from 'child_process'

async function curl (url: string) {
  return await new Promise(async resolve => {
    const command = `curl "${url}" -L --connect-timeout 10 --max-time 12`
    exec(command, { maxBuffer: 1024 * 1024 * 10 }, (err, stdout, stderr) => {
      if (err) {
        console.error(err, stderr)
        resolve('')
      } else {
        resolve(stdout)
      }
    })
  })
}

export default curl