Retrieving all documents of all collections from a database in MongoDB with PyMongo

Question

I have a MongodDB database (mydbase) with different collections in it (coll1, coll2, coll3). I want to collect all documents of all collections in a structure of list (=database) of lists (=collections) of dicts (documents) with PyMongo. I tried the following:

[list(db.coll.find({})) for coll in db.collection_names()]

but it returns a list of empty lists. Which is kind of a weird, because if I query only one collection in a similar way:

list(db.coll1.find({}))

that gives a populated list as hoped for. What is the problem here?

Answer 1

test = [ list(db[coll].find({})) for coll in db.collection_names() ]

coll is a variable, so we need to use db[coll] instead of db.coll

for example, for a db with a collection named 'knights'

db.knights.find_one({}) # correctly from 'knights'
n = 'knights'
db.n.find_one({}) # get one document from 'n'
db[n].find_one({}) # correctly from 'knights'
db['n'].find_one({}) # from 'n' again

Answer 2

You could use this:

[[record for record in db[collection_name].find({})] for collection_name in db.collection_names()]

I would read this in the following way:

[something for collection_name in db.collection_names()]

We use db.collection_names() to obtain an iterable representing the names of the collections in our db.

We then iterate over that iterable, putting the current collection name in the variable collection_name . With that collection_name variable we do something .

This will form the outer list of your desired outcome.

Now, let's address the inner lists.

[record for record in db[collection_name].find({})]

First, let's identify the iterable. That is, db[collection_name].find({}) . It's the combination of the two, db[collection_name] to access the collection and .find({}) to get an iterable.

With the collection_name variable, from the outer list, we access the current collection. We can think of db[collection_name] as the current collection we are extracting records from.

From that collection, we find all records, .find({}) and form a list with them.

Also, keep in mind, you are loading everything in memory. Depending on the size of your data base, this may not be a very good idea!