Return MongoDB collection with AWS Lambda and API Gateway

Question

I have a function in AWS Lambda written in Python.

I am trying to extract documents from a collection in MongoDB with pymongo .

I thought it was quite simple, but I seem to get problems (maybe because of ObjectID types).

I am simply trying to do

from pymongo import MongoClient

def lambda_handler(event, context):

  client = MongoClient(MONGODB_URI)

  db = client[DB_NAME]

  return db.users.find({})

but I get the error

{errorMessage= is not JSON serializable, errorType=TypeError, stackTrace=[["\/var\/lang\/lib\/python3.6\/json\/__init__.py",238,"dumps","**kw).encode(obj)"],["\/var\/lang\/lib\/python3.6\/json\/encoder.py",199,"encode","chunks = self.iterencode(o, _one_shot=True)"],["\/var\/lang\/lib\/python3.6\/json\/encoder.py",257,"iterencode","return _iterencode(o, 0)"],["\/var\/runtime\/awslambda\/bootstrap.py",110,"decimal_serializer","raise TypeError(repr(o) + \" is not JSON serializable\")"]]}

It does work if I use return bson.json_util.dumps(db.users.find({})) , but why should it be necessary?

As far as I understand, lambda functions always return json, so I don't understand why I have to use bson.json_util .

Also, when I use this function, I don't get normal ObjectID types, but instead I get

[
  {"_id": {"$oid": "59aed327f25c0f0ca8f94ae1"}, "name": ...},
  ...
]

although I wanted something like

[
  {"_id": "59aed327f25c0f0ca8f94ae1", "name": ...},
  ...
]

Answer 1

Your issue is due to pymongo not returning straight JSON strings. An example of how to handle this can be found here -

How do I turn MongoDB query into a JSON?

It should be noted that API Gateway expects to return JSON unless configured otherwise. https://aws.amazon.com/blogs/compute/binary-support-for-api-integrations-with-amazon-api-gateway/