等待嵌套的...循环

Question

  async traverse(url) {
    const ts = new TournamentScraper()
    const ms = new MatchScraper()
    const results = []
    const tournaments = await ts.run(url)
    for(let href of tournaments.map(t => t.href)){
      let matches = await ms.run(href)
      let pages = ms.getPages()
      let seasons = ms.getSeasons()
      //console.log(pages)
      //console.log(seasons)
      results.push(matches)
      for(let href of pages) {
        //console.log(href)
        matches = await ms.run(href)
        //console.log(matches)
        results.push(matches)
      }
    }

    return results
  }

TournamentScraper returns an array of objects, which typically looks like this:

{name: 'Foo', href: 'www.example.org/tournaments/foo/'}

The link points to the tournament's last season's first page. This page contains the links to the other seasons and a paginator (if any).

MatchScraper's run returns some data, and sets the instance's dom property. getPages() and getSeasons() consumes this property and each returns an array of links.

The problem that results contains only the first batch of matches. I can see the 2nd page's matches in the console log, but they are not in the results array when traverse returns.

I found this rule which is against await in for loop. The problem, that I have to wait for ms.run(href) , because it sets dom, and getPages() and getSeasons() needs it to be set, to extract the needed links.

Answer 1

I think this should work. It utilizes Promise all rather than for loops

const run = href => ms.run(href);

async function getMatches(href) {  
  const out = [];
  const matches = await run(href);
  const pages = ms.getPages();

  out.push(matches);

  if(pages.length) {
    const pageResults = await Promise.all(pages.map(href => run(href)));
    out.push(...pageResults);
  }

  return out;
}

async function traverse(url) {
  const ts = new TournamentScraper();
  const ms = new MatchScraper();
  const tournaments = await ts.run(url)
  const matches = await Promise.all(tournaments.map(t => getMatches(t.href)));
  return matches.reduce((a, b) => { 
    a.push(...b);
    return a;
  }, []);
}