[Gatsby][GraphQL]: running a query after retrieving filter parameters from another query

Question

Quick background:

I've got a listings project with around 40 cities and 16 regions that I'm targeting. I'm programatically creating a search results page for each city:

example.com/london, example.com/paris etc...

Then, I need each city page to have a query to retrieve listings that are only related to that city.

As of now, I'm querying same listings on each search page and then in the component I'm filtering the results on the client. The problem with that solution is that I'm loading thousands of listings on each page with page-data.json that I don't need.

I don't expect the listings to exceed few thousands that's why I don't want to add apollo to query directly from the client. I'd like all pages to be ssr'd. Filtering of results and pagination will be done via component and filtering of the array of results once the page loads.

The way I imagined that was:

1. Run a query to retrieve list of cities
2. For each city retrieved run a actual page query with cityId as filter parameter. For performance purposes, I'd like that to happen in gatsby-node.js and not pass cityId to the pageContext and then run a pageQuery from the page.js (which for some reason I couldn't make that work either)

Here's my gatsby-node.js

 const path = require('path') function slugify(str) { str = str.replace(/^\s+|\s+$/g, ''); // trim str = str.toLowerCase(); // remove accents, swap ñ for n, etc var from = "ãàáäâąęẽèéëêćìíïîõòóöôùúüûñńçłśżź·/_,:;"; var to = "aaaaaaeeeeeeciiiiooooouuuunnclszz------"; for (var i=0, l=from.length; i<l; i++) { str = str.replace(new RegExp(from.charAt(i), 'g'), to.charAt(i)); } str = str.replace(/[^a-z0-9 -]/g, '') // remove invalid chars.replace(/\s+/g, '-') // collapse whitespace and replace by -.replace(/-+/g, '-'); // collapse dashes return str; }; exports.createPages = async ({ graphql, actions }) => { const { createPage } = actions; const listingQueryResults = await graphql(` query { allDatoCmsListing { nodes { company { cities { cityName region { regionName } } companyName address logo { fixed(imgixParams: {w: "128", h: "128", fit: "fillmax"}) { src } } #Companywide Terms insurancePolicy otherInsuranceTerms pricePerKm minAge deposit bookingPhoneNumber displayPhoneNumber bookingEmail } featuredInCountry monthlyPrice listingTitle pricesIncludeVat id originalId featuredImage { fluid(imgixParams: {fit: "crop", w: "800", h: "600", crop: "focalpoint"}) { aspectRatio base64 height sizes src srcSet tracedSVG width } originalId } gallery { fluid { width tracedSVG srcSet src sizes height base64 aspectRatio } } featuredInCity featuredInRegion listingDescription make { makeName } spec seats topSpeed transmission { transmissionType } weekendLimit weekendNoDepositPrice weekendPrice weeklyLimit weeklyNoDepositPrice weeklyPrice acceleration collectionDropoff color { colorName colorValue } dailyLimit dailyNoDepositPrice dailyPrice doors engine { engineType } engineSize horsepower monthlyLimit monthlyNoDepositPrice noDepositPricingAvailable #Listing Terms applyCompanywideTerms insurancePolicy otherInsuranceTerms pricePerKm minAge deposit listingApproved } } } `); const listingTemplate = path.resolve(`src/templates/listing.js`); listingQueryResults.data.allDatoCmsListing.nodes.forEach(node => { createPage({ path: `/oferta/${node.originalId}-${slugify(node.listingTitle)}`, component: listingTemplate, context: { listing: node } }); }); const queryResults = await graphql(` query { allDatoCmsCity { nodes { cityName cityCase id } } allDatoCmsRegion { nodes { regionName regionCase id } } } `); const searchTemplate = path.resolve(`src/templates/search.js`); queryResults.data.allDatoCmsCity.nodes.forEach(node => { createPage({ path: `/${slugify(node.cityName)}`, component: searchTemplate, context: { search: node, } }); }); queryResults.data.allDatoCmsRegion.nodes.forEach(node => { createPage({ path: `/${slugify(node.regionName)}`, component: searchTemplate, context: { search: node } }) }) const emptySearch = { cityName: null, regionName: null } createPage({ path: `/cala-polska`, component: searchTemplate, context: { search: emptySearch } }) };

I guess the more precised question is:

1. What's the best way to achieve the above. That is to get all cities & regions
2. Loop through cities & regions and query each city & region separately as opposed to running the exact same query and getting results for all cities/regions on a specific city/region page?

Answer 1

So I've spent some hours on this. And I found a solution that works wonders for me. Went from 5 page queries / second to hundreds. That's because I query only once for all cities, regions and listings.

Then I wrote a filter function that's just chain of

FilterResults = (arr, params) => (
   arr.filter(/* city filter / if filter is null -> return true and move to the next*/)
      .filter(/*region filter - same as city */)
)

That function returns array of listings.

We loop the city results as follows:

query.allDatoCmsCity.nodes.forEach(node => {

params = {
  city: node.id
}

results = FilterResults(query.allDatoCmsListing.nodes, params)

// Then our typical gatsby create page

createPage({
      path: `/${slugify(node.cityName)}`,
      component: searchTemplate,
      context: {
        search: node,
        listings: results
      }
    });
})

This makes us query only once for all listings instead of 56 times for all listings (because I was using a page query in the template, that would essentially be called on every createPage)

Not only is this cleaner code, it's also much more performant. Hope I helped someone as much as I did myself;)