Is there a way to fill the JSON object before resolving the promise

Question

the code first gets all the urls from the database. and in parseText i'm trying to parse all the ur's and put them in an Json object for later reference.

i've tried running the for loop with async/await, but that didnt give me the expected result.

let parseText = function(dbresult) {
    return new Promise((resolve, reject) => {
    let textObj = {}

    for(i=0; i < dbresult; i++) {
       Mercury.parse(dbresult[i].url, {headers: {Cookie: 'name=Bs', 'User-Agent': 'Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)', },})
       .then((result) => {
           textObj[dbresult[i].id] = result.excerpt;
      });
    }
    resolve(textObj);
  })
}


fetchLinks.then(function(result) {
    return parseText(result);
  }).then(function(result) {
  console.log(result); //gives back {}
  //do something with json object in next promise
  return writeTextToDb(result); //not written yet
})

the desired output should look something like {1234 : {text: some parsed text}}, but all i keep getting is an empty object

Answer 1

There are multiple things to tackle in your code, so let's go step by step:

• dbresult seems to be an array, judging by your use of dbresult[i] , but you also have a i < dbresult condition that implies it's an integer. I'm going to asume you meant i < dbresult.length .
• You are using new Promise(...) in a situation where you are already dealing with promises. You should never use this pattern unless you don't have an alternative, and always try to chain .then calls and return their results (which are always also promises).
• You seem to have failed to understand that the callback passed to .then will always run asynchronously, after the rest of the code runs. This is the reason your object comes out empty: the resolve function is called before any of the requests have time to complete.

Now, loops and promises don't mix very well, but there are ways of dealing with them. What you need to anderstand is that, with loops, what you want is to chain promises . There are mostly two ways of chaining promises in this way: the imperative way and the functional way.

I'm going to focus on the parseText function and omit irrelevant details. This is what you would do for a fully imperative solution:

function parseText (dbresult) {
    // although the contents of the object change, the object doesn't,
    // so we can just use const here
    const textObj = {};

    // initialize this variable to a dummy promise
    let promise = Promise.resolve();

    // dbresult is an array, it's clearer to iterate this way
    for (const result of dbresult) {
       // after the current promise finishes, chain a .then and replace
       // it with the returned promise.  That will make every new iteration
       // append a then after the last one.
       promise = promise
         .then(() => Mercury.parse(result.url, {...}))
         .then((response) => (textObj[result.id] = response.excerpt));
    }

    // in the end, the promise stored in the promise variable will resolve
    // after all of that has already happened.  We just need to return the
    // object we want to return and that's it.
    return promise.then(() => textObj);
}

I hope the comments help. Again, working with promises in a loop sucks.

There are two ways of doing this in an easier way, though! both use the functional methods of arrays. The first one is the easiest, and it's the one I'd recommend unless the array is going to be very big. It makes use of .map and Promise.all , two powerful allies:

function parseText (dbresult) {
    const textObj = {};

    // create an array with all the promises
    const promises = dbresult.map(result => Mercury.parse(result.url, {...})
        .then((response) => (textObj[result.id] = response.excerpt)))
    );

    // await for all of them, then return our desired object
    return Promise.all(promises).then(() => textObj);
}

Note: bluebird users can make this even better using Promise.map and passing a concurrency value. That is actually, in my opinion, the best solution, but I want to stick with vanilla here.

The main problem with this solution, though, is that all request will be started at once . This might mean that, for very large arrays, some requests just wait in a queue or you exhaust the socket limit of the process, depending on the implementation. In any case, it's not ideal, but for most situations will work.

The other functional solution consists of replicating the imperative one using .reduce instead of a for ... of loop, and it's implemented at the end of the answer, more as a curiosity than anything else, as I think it's a bit too "clever code".

In my opinion, the best way of solving this is to just use async/await and forget promises altogether . You can just write your loop normally in this case and simply put await where appropriate:

async function parseText (dbresult) {
    const textObj = {};

    for (const result of dbresult) {
        // here just await the request, then do whatever with it
        const response = await Mercury.parse(result.url, {...}))
        textObj[result.id] = response.excerpt;
    }

    // thanks to await, here we already have the result we want
    return textObj;
}

That's it, it's that simple.

---

And now for what I consider to be the "clever" solution, using only .reduce :

function parseText (dbresult) {
    const textObj = {};
    return dbresult.reduce(
        (prom, result) => prom
            .then(() => Mercury.parse(result.url, {...}))
            .then((response) => (textObj[result.id] = response.excerpt)),
        Promise.resolve()
    ).then(() => textObj);
}

If it's not immediately clear what it does, that's normal. This does exactly the same thing as the original imperative then -chaining one does, just using .reduce instead of a manual for loop.

Note that I personally wouldn't necessarily do this, as I think it's a bit too "clever" and takes some time to parse mentally. If implementing something like this ( then -chaining using .reduce is incredibly useful, even if a bit confusing) please add a comment explaining why you did this, what it is, or something that can help other developers understand ir at first glance.