Why doesn't 'await' wait for axios request completes?

Question

I tried to read HTTP response with axios and parse JSON in stream mode with stream-json` so I can fully fit my database on demand. It works well but if I try to close database connection everything will be crashed because the connection will be closed too soon. The problem is: await doesn't wait for the extract_coins function to complete (even if it's returning the promise) and close database connection in a final scope.

const main = async() => {
   const dbcfg = config.get('db.coins');
   const db = await coins_db_configure(dbcfg);
   try {
      console.log('Connected to the database and created coins table!');

      await extract_coins('some_url_which_gives_correct_json', coins_emitter);
   }
   catch(e){
      console.error(e);
   }
   finally {
      await db.close();
   }
   };

main();

extract_coins:

module.exports = async function extract_coins(url, emitter){
    return await axios({
        method: 'get',
        url: url,
        responseType: 'stream'
    }).then((res) => {
        const pipeline = chain([
            res.data,
            parser(),
            pick({filter: 'data'}),
            streamArray()
        ]);
        pipeline.on('data', data => {
            emitter.emit('coin_extracted', data.value);
        });
        pipeline.on('end', () => console.log("All the coins were successfully passed!"));
    });
};

Answer 1

As the code related to the asynchronous pipeline is not promisified, you currently have no way to return a promise that resolves on getting the "end" event.

Your await does wait for the promise to resolve, but your then callback returns undefined , and so that resolves the promise at that very moment, long before the end event is broadcast.

So change this:

then((res) => {
    const pipeline = chain([
        res.data,
        parser(),
        pick({filter: 'data'}),
        streamArray()
    ]);
    pipeline.on('data', data => {
        emitter.emit('coin_extracted', data.value);
    });
    pipeline.on('end', () => console.log("All the coins were successfully passed!"));
});

To this:

then((res) => new Promise((resolve) => {
    const pipeline = chain([
        res.data,
        parser(),
        pick({filter: 'data'}),
        streamArray()
    ]);
    pipeline.on('data', data => {
        emitter.emit('coin_extracted', data.value);
    });
    pipeline.on('end', () => {
        console.log("All the coins were successfully passed!");
        resolve();
    });
}));

Answer 2

Actually, you used ES6+ way but not in a common way, this causes this issue but, definitely, using await alongside then is exactly a fault, you should write them just like following codes:

const main = async () => {
   try {
     const dbcfg = config.get('db.coins');
     const db = await coins_db_configure(dbcfg);
     console.log('Connected to the database and created coins table!');
     await extract_coins('some_url_which_gives_correct_json', coins_emitter);

   } catch (e) {
      console.error(e);

   } finally {
      await db.close();

   }
};

main();

As you see, I put all of the codes inside try block, because using async/await means we pretend to write a sync code but it is async in fact, so we should put all codes, especially asynchronous lines inside try block. after putting them inside try block, just because of async flag behind of () JavaScript interpreter wait for each line to finish, if each line has error next lines won't run and interpreter falls into catch block, finally runs in all cases as you now.

Ok now back to the main issue, the extract_coins function:

export default (async function extract_coins(url, emitter) {
  try {
    const res = await axios({
      method: 'get',
      url: url,
      responseType: 'stream'
    });
    const pipeline = chain([
      res.data,
      parser(),
      pick({filter: 'data'}),
      streamArray()
    ]);
    await pipeline.on('data', data => {
      emitter.emit('coin_extracted', data.value);
    });
    await pipeline.on('end', () => console.log("All the coins were successfully passed!"));
    // HERE: return what you want

  } catch (e) {
    throw e;

  }

});

This is the cause of the issue, you should pass the promise of extract_coins function with the new EcmaScript method, not using callback functions.

If I wan in your place I would write the extract_coins like below:

const extract_coins = async (url, emitter) => {
  try {
    const res = await axios({
      method: 'get',
      url: url,
      responseType: 'stream'
    });
    const pipeline = chain([
      res.data,
      parser(),
      pick({filter: 'data'}),
      streamArray()
    ]);
    await pipeline.on('data', data => {
      emitter.emit('coin_extracted', data.value);
    });
    await pipeline.on('end', () => console.log("All the coins were successfully passed!"));
    // HERE: return what you want

  } catch (e) {
    throw e;

  }

};

export default extract_coins;