How to use Mongoose with GraphQL and DataLoader?

Question

I am using MongoDB as my database and GraphQL . I am using Mongoose for my model. I realised my GraphQL queries are slow because the same documents are being loaded over and over again. I would like to use DataLoader to solve my problem, but I don't know how.

Example

Let's say I have the following schema, describing users with friends :

// mongoose schema
const userSchema = new Schema({
  name: String,
  friendIds: [String],
})

userSchema.methods.friends = function() {
  return User.where("_id").in(this.friendIds)
}

const User = mongoose.model("User", userSchema)

// GraphQL schema
const graphqlSchema = `
  type User {
    id: ID!
    name: String
    friends: [User]
  }

  type Query {
    users: [User]
  }
`

// GraphQL resolver
const resolver = {
  Query: {
    users: () => User.find()
  }
}

Here is some example data in my database :

[
  { id: 1, name: "Alice", friendIds: [2, 3] },
  { id: 2, name: "Bob", friendIds: [1, 3] },
  { id: 3, name: "Charlie", friendIds: [2, 4, 5] },
  { id: 4, name: "David", friendIds: [1, 5] },
  { id: 5, name: "Ethan", friendIds: [1, 4, 2] },
]

When I do the following GraphQL query :

{
  users {
    name
    friends {
      name
    }
  }
}

each user is loaded many times. I would like each user Mongoose document to be loaded only once.

What doesn't work

Defining a "global" dataloader for fetching friends

If I change the friends method to :

// mongoose schema
const userSchema = new Schema({
  name: String,
  friendIds: [String]
})

userSchema.methods.friends = function() {
  return userLoader.load(this.friendIds)
}

const User = mongoose.model("User", userSchema)

const userLoader = new Dataloader(userIds => {
  const users = await User.where("_id").in(userIds)
  const usersMap = new Map(users.map(user => [user.id, user]))
  return userIds.map(userId => usersMap.get(userId))
})

then my users are cached forever rather than on a per request basis.

Defining the dataloader in the resolver

This seems more reasonable : one caching mechanism per request.

// GraphQL resolver
const resolver = {
  Query: {
    users: async () => {
      const userLoader = new Dataloader(userIds => {
        const users = await User.where("_id").in(userIds)
        const usersMap = new Map(users.map(user => [user.id, user]))
        return userIds.map(userId => usersMap.get(userId))
      })
      const userIds = await User.find().distinct("_id")
      return userLoader.load(userIds)
    }
  }
}

However, userLoader is now undefined in the friends method in Mongoose schema. Let's move the schema in the resolver then!

// GraphQL resolver
const resolver = {
  Query: {
    users: async () => {
      const userLoader = new Dataloader(userIds => {
        const users = await User.where("_id").in(userIds)
        const usersMap = new Map(users.map(user => [user.id, user]))
        return userIds.map(userId => usersMap.get(userId))
      })
      const userSchema = new Schema({
        name: String,
        friendIds: [String]
      })
      userSchema.methods.friends = function() {
        return userLoader.load(this.friendIds)
      }
      const User = mongoose.model("User", userSchema)
      const userIds = await User.find().distinct("_id")
      return userLoader.load(userIds)
    }
  }
}

Mh ... Now Mongoose is complaining on the second request : resolver gets called again, and Mongoose doesn't like 2 models being defined with the same model name.

"Virtual populate" feature are of no use, because I can't even tell Mongoose to fetch models through the dataloader rather than through the database directly.

Question

Has anyone had the same problem? Does anyone have a suggestion on how to use Mongoose and Dataloader in combination? Thanks.

Note: I know since my schema is "relational", I should be using a relational database rather than MongoDB. I was not the one to make that choice. I have to live with it until we can migrate.

Answer 1

Keep your mongoose schema in a separate module. You don't want to create your schema each request -- just the first time the module is imported.

const userSchema = new Schema({
  name: String,
  friendIds: [String]
})
const User = mongoose.model("User", userSchema)

module.exports = { User }

If you want, you can also export a function that creates your loader in the same module. Note, however, that we do not want to export an instance of a loader, just a function that will return one.

// ...
const getUserLoader = () => new DataLoader((userIds) => {
  return User.find({ _id: { $in: userIds } }).execute()
})
module.exports = { User, getUserLoader }

Next, we want to include our loader in the context. How exactly this is done will depend on what library you're using to actually expose your graphql endpoint. In apollo-server , for example, context is passed in as part of your configuration.

new ApolloServer({
    typeDefs,
    resolvers,
    context: ({ req }) => ({
        userLoader: getUserLoader()
    }),
})

This will ensure that we have a fresh instance of the loader created for each request. Now, your resolvers can just call the loader like this:

const resolvers = {
  Query: {
    users: async (root, args, { userLoader }) => {
      // Our loader can't get all users, so let's use the model directly here
      const allUsers = await User.find({})
      // then tell the loader about the users we found
      for (const user of allUsers) {
        userLoader.prime(user.id, user);
      }
      // and finally return the result
      return allUsers
    }
  },
  User: {
    friends: async (user, args, { userLoader }) => {
      return userLoader.loadMany(user.friendIds)
    },
  },
}