setTimeout in call to API endpoint not returning values

Question

I'm attempting to make a call to a separate endpoint within the Yelp API to extract business description info and hours of operation. I defined the initial POST route in Express and, using ejs, am able to display data when called for the first endpoint. I need to pass business IDs from an array created in my first endpoint to a call to the second, however I am getting TooManyRequests responses and therefore had to make use of setTimeout ; this is where I'm running into problems

While setTimeout will allow me to console.log the returned results from the second call, it doesn't actually allow me to return any data to present to the page. No errors are thrown and I am not being returned undefined , further adding to my confusion.

I attempted to wrap setTimeout into an IIFE both inside and outside of my for-loop which yielded the same results as before. I briefly considered implementing Promises but was seeing if this could be accomplished solely using setTimeout . Not sure if I'm using the proper approach here but any help or clarification would be greatly appreciated.

app.js

express = require("express");
var app = express(); 
var path = require("path"); 
var yelp = require("yelp-fusion");
var request = require("request");
var bodyParser = require("body-parser");

app.use(express.static(__dirname + '/public')); 
app.use(bodyParser.urlencoded({extended: true}));
app.set("view engine", "ejs");

let client = yelp.client("API_KEY_HID");

app.get("/", function(req,res){
res.render("landing");
});

app.post("/", function(req, res){
  client.search({
  term: 'coffee',
  location: 'Oakland',
  limit: 10
}).then(response => {
    var businesses = response.jsonBody.businesses;
    var idArray = [];  
    var openHours;
    var id = businesses.forEach(el => {
    idArray.push(el.id); 
    });
for(var x = 0; x < businesses.length; x++){
    var delay = 1 * 1000;
    setTimeout(function(x){
        client.business(idArray[x]).then(response => {
        var hours = response.jsonBody.hours.map(el => {
            return el.open;
        });
        openHours = hours.map(lo => {
                return lo[x].start;
        }); 
        return openHours; 
        });  
    },delay*x,x)
}
res.render('search', {
    hour: openHours
});
}).catch(e => {
console.log(e);
});
});

app.listen(3000); 

Answer 1

Here is the reason you are not seeing any data:

the then function actually returns a Promise . It resolves with the return value of the function that's passed into it , which in this case is openHours .

To be more expansive, client.business(id) is a Promise . client.business(id).then(resolvingFunction) is also a Promise . As you know, when client.business(id) resolves, it executes the function passed into then . What you didn't know is that client.business(id).then also needs to resolve . The way you wrote it, it resolves with the value openHours . In order to execute, it needs it's own .then . It would need to be client.business(id).then(...).then(openHours=>{do something with openHours}) . You don't need two then s in this case, as long as instead of returning openHours in your then , you just do something with it. The smallest change possible to fix this issue would be moving res.render to the line that return openHours currently is. That would be kind of messy though. Alternatively you could do this with multiple promises or using async / await .

Hope this helps.

Answer 2

There are lots of issues with your code. For starters, setTimeout() is asynchronous and non-blocking. That means your for() loop runs to completeion setting a bunch of timers, you call res.render('search', {hour: openHours}) before ANY of the setTimeout() callbacks have been called and thus openHours is still empty.

Then, to solve the problem TooManyRequests error you're getting, you will have to redesign how you make requests and how you monitor when they are all done. To do that optimally, you would have to know what your actual request limits are from the host you're requesting data from. You need to know whether you are limited to a certain number of parallel requests in flight at the same time, limited to a certain number of requests within a certain time period (like 1 request/sec) or whether there is some other rule.

Without knowing how you're actually limited, you could design a serial request system (one request at a time with an adjustable delay between requests). Chances are you could tune that to work with whatever limits your host is enforcing (unless you're just making too many total requests). If you knew the actual rules, you could design more efficient code. But, without knowing the rules, here's a tunable, serialized approach.

In addition, there were a number of other unclear portions of your code as it wasn't clear exactly what you were trying to accumulate in all your setTimeout() callbacks. Each one was setting openHours to a new value (overwriting the previous value) so I couldn't tell exactly what you want that data to look like. You will have to fill in that portion in the code below.

This serializes all your requests, one after another with a settable delay between them. You can hopefully tune that to slow down your requests to something the host is OK with.

This code uses a .reduce() design pattern to serialize promises that works in any environment. There are many other design patterns for serializing promise-base operations too. If you have an ES7 environment, then async/await in a for loop can be a simple way to go too.

var express = require("express");
var app = express();
var path = require("path");
var yelp = require("yelp-fusion");
var request = require("request");
var bodyParser = require("body-parser");

app.use(express.static(__dirname + '/public'));
app.use(bodyParser.urlencoded({
    extended: true
}));
app.set("view engine", "ejs");

// utility delay function that returns a promise
function delay(t, v) {
    return new Promise(resolve => {
        setTimeout(resolve.bind(v), t);
    });
}

let client = yelp.client("API_KEY_HID");

app.get("/", function(req, res) {
    res.render("landing");
});

app.post("/", function(req, res) {
    client.search({
        term: 'coffee',
        location: 'Oakland',
        limit: 10
    }).then(response => {
        var businesses = response.jsonBody.businesses;
        var idArray = businesses.map(el => el.id);

        // set your delay value here between reqeusts (here it is set to 500ms)
        const delayBetweenRequests = 500;

        return idArray.reduce((id, p, i) => {
            return p.then((array) => {
                return client.business(id).then(response => {
                    // not sure what you want to get out of this response and add to the array of openHours
                    // that we're accumulating
                    array.push(something here to add to the results);
                    // return delay promise so the next request will wait
                    return delay(delayBetweenRequests, array);
                });
            });
        }, Promise.resolve([])).then(openHours => {
            res.render('search', {hour: openHours});
        });

    }).catch(e => {
        console.log(e);
        res.sendStatus(500);
    });
});

app.listen(3000);