Why does res.send() not display results in my index.html page when I place it inside of my Promise.then statement?

Question

I've been trying to just extract and scrape the number of COVID cases off of a website and display them on an index.html page, using puppeteer to scrape the data from the official COVID website. I have this code for my API in my index.js file:

const func = require('./scrapers');

app.get('/creators', async (req, res) => {
  const numC =  func.scrapeCases('https://covid19.ca.gov/state-dashboard/');
  var myFinal = numC.toString();
  numC.then(result => {
    const setter = result.toString();
    myFinal = setter;
    console.log(myFinal)
    //res.send('6')
    res.send(myFinal)
  });
})

The function "scrapeCases" is in a file called scrapers.js in the same directory as index.js. It returns the jsonValue() of the number I'm trying to extract from the COVID website (number of cases), which in reality gives me a Promise return value, which is why I also used toString() to change that.

When I do console.log(myFinal) as given, the proper number of cases is shown in the terminal as if it were a regular string. But when I do res.send(myFinal), it doesn't show up in my index.html page, but if I comment res.send(myFinal) and do res.send('6') as an arbitrary example, the 6 is displayed (after a short delay of time, not sure why). Why is it that it doesn't work with res.send?

Also, if I do this instead:

app.get('/creators', async (req, res) => {
  const numC =  await func.scrapeCases('https://covid19.ca.gov/state-dashboard/');
  var myFinal = numC.toString();
  console.log(myFinal)
  //res.send('6')
  res.send(myFinal)
})

Here I add "await" to my func.ScrapeCases statement since I think this is another way I can get the value of the Promise of numC. Again, the console.log(myFinal) gives me the proper amount of cases as a simple number with commas in between (eg 1,366,234), and res.send('6') again displays on my index.html if I uncomment it, but res.send(myFinal) does not display anything.

Any help is appreciated thank you!

Answer 1

By wrapping res.send(), it is modified to be executed after the async function is executed.

This is the code that created and solved the same situation as you.

const express = require('express');
const app = express();

const asyncWrapper = (fn) => {
    return (req, res, next) => {
      return Promise.resolve(fn(req))
        .then((result) => res.send(result))
        .catch((err) => next(err))
    }
  }

const count = async () => {
    let count = 0;
    for(let i = 0; i < 1000000; ++i){
        count++;
    }

    return count.toString();
}

app.get('/creators', asyncWrapper(count));

app.listen(3000, () => {
    log(`API Server listening on port 3000!`);
});

Probably your code will be like this (I haven't tested it. May need to be modified)

const func = require('./scrapers');

const asyncWrapper = (fn) => {
    return (req, res, next) => {
      return Promise.resolve(fn(req))
        .then((result) => res.send(result))
        .catch((err) => next(err))
    }
  }

const count = async () => {
    let numC = await func.scrapeCases('https://covid19.ca.gov/state-dashboard/');
    return numC.toString();
}

app.get('/creators', asyncWrapper(count));