How do you save a Python object into pymongo? (i.e. what method/return value do I need to override)

Question

Let's say I have a class: TestClass . This class is slotted.

class TestClass(object):
    __slots__ = ('_id', 'value1', 'value2',)

So we create an object.

test = TestClass()
test.key1 = 'val1'
test.key2 = 'val2'

Great! Now what I would like to do is insert test into a MongoDB instance.

db.test_collection.insert(test)

Uh oh.

TypeError: 'TestClass' object is not iterable

Ok, let's make that iterable.

class TestClass(object):
    __slots__ = ('_id', 'key1', 'key2',)

    def __iter__(self):
        yield from dict(
            (slot, self.__getattribute__(slot))
            for slot in self.__slots__).items()

test = TestClass()
test.key1 = 'val1'
test.key2 = 'val2'

for i in test:
    print(i)

// result
// ('key1', 'val1')
// ('key2', 'val2')

db.test_collection.insert(test)

This gives me: doc['_id'] = ObjectId() // TypeError: 'tuple' object does not support item assignment .

Further, let's say I have a composition of objects...

test = TestClass()
test.key1 = 'val1'
test.key2 = TestClass()

Would the pymongo encoder be able to encode test.key2 when saving test ?

EDIT: I'm okay not saving the object directly and calling a function on the object like test.to_document() , but the goal is to have composite fields (eg test.key2 ) become a dict so that it can be saved.

Answer 1

First I would ask myself, why is it important to store the object as is in the Mongodb. What is it that I want to achieve here?

If the answer to the above question is: "Yes, there is a valid reason for this". The way to store objects is to first serialize them, and then store that serialization.

The pickle module does that for you.

import pickle

test = TestClass()
dumped = pickle.dumps(test)
db.objects.insert_one({"data": dumped, "name": 'test'})

It's worth noting that because these are python objects and if a user in anyway has the possibility to insert a pickled object into the database and you at some point unpickles that object it would pose a security threat to you.

Answer 2

As you are trying to insert an individual object rather than a series of them, the first error message you got, suggesting that PyMongo is expecting an iterable, is perhaps misleading. insert can take either a single document (dictionary) or an iterable of them.

You can convert the object to a dict with a function like this :

def slotted_to_dict(obj):
    return {s: getattr(obj, s) for s in obj.__slots__ if hasattr(obj, s)}

Then

db.test_collection.insert(slotted_to_dict(test))

should work, but is deprecated . So

db.test_collection.insert_one(slotted_to_dict(test))

is probably better.

Note that if you are frequently converting the slotted object to a dict, then you might be losing any performance benefit from using slots. Also, the "_id" attribute will not be set in the test object, and you might need an awkward solution like this to save the ID:

test._id = db.test_collection.insert_one(slotted_to_dict(test))