如何为 python 单元测试模拟 mongodb？

Question

I am using mock module for Python 2.7 to mock my other functions and using

unittest for writing unit tests.

I am wondering if mocking the MongoDB is different than using mock functionality ( mock.patch a function that is being called?) Or I need to use another different package for that purpose?

I do not think I want to have a test mongodb instance running. All I want is some tempo data and being able to call pymongo functionality. I am just a bit lost in thinking of is there a way to write a mock for a module (like pymongo ), or anything is achievable by mock module.

So appreciate if you could provide an example or tutorial on this.

Code to Test

from pymongo import MongoClient

monog_url = 'mongodb://localhost:27017'
client = MongoClient(monog_url)
db = client.db

class Dao(object):
   def __init__(self):
      pass

   def save(self, user):
      db_doc = {
        'name': user.name,
        'email': user.email
      }
      db.users.save(db_doc)

   def getbyname(self, user):
      db_doc = {
        'name': user.name,
      }
      return db.users.find(db_doc)

To test this, I do not really want a test mongodb up and running, But also. I think I do not want to mock db.userssave and db.users.find because I want to actually be able to retrieve the data that I saved and make sure it is in the db. I think I need to create some fixtures per models that are in my memory and work with them. Just do I need an external tool to do so?

I am thinking of keeping some fake data like this, just do not know how to properly deal with it.

users = { 
    {'name' : 'Kelly', 'email' : 'kelly@gmail.com'},
    {'name': 'Sam', 'email': 'sam@gmail.com'}
}

Answer 1

I am using mock module for Python 2.7 to mock my other functions and using

unittest for writing unit tests.

I am wondering if mocking the MongoDB is different than using mock functionality ( mock.patch a function that is being called?) Or I need to use another different package for that purpose?

I do not think I want to have a test mongodb instance running. All I want is some tempo data and being able to call pymongo functionality. I am just a bit lost in thinking of is there a way to write a mock for a module (like pymongo ), or anything is achievable by mock module.

So appreciate if you could provide an example or tutorial on this.

Code to Test

from pymongo import MongoClient

monog_url = 'mongodb://localhost:27017'
client = MongoClient(monog_url)
db = client.db

class Dao(object):
   def __init__(self):
      pass

   def save(self, user):
      db_doc = {
        'name': user.name,
        'email': user.email
      }
      db.users.save(db_doc)

   def getbyname(self, user):
      db_doc = {
        'name': user.name,
      }
      return db.users.find(db_doc)

To test this, I do not really want a test mongodb up and running! But also, I think I do not want to mock db.userssave and db.users.find because I want to actually be able to retrieve the data that I saved and make sure it is in the db. I think I need to create some fixtures per models that are in my memory and work with them. Just do I need an external tool to do so?

I am thinking of keeping some fake data like this, just do not know how to properly deal with it.

users = { 
    {'name' : 'Kelly', 'email' : 'kelly@gmail.com'},
    {'name': 'Sam', 'email': 'sam@gmail.com'}
}

Answer 2

I am using mock module for Python 2.7 to mock my other functions and using

unittest for writing unit tests.

I am wondering if mocking the MongoDB is different than using mock functionality ( mock.patch a function that is being called?) Or I need to use another different package for that purpose?

I do not think I want to have a test mongodb instance running. All I want is some tempo data and being able to call pymongo functionality. I am just a bit lost in thinking of is there a way to write a mock for a module (like pymongo ), or anything is achievable by mock module.

So appreciate if you could provide an example or tutorial on this.

Code to Test

from pymongo import MongoClient

monog_url = 'mongodb://localhost:27017'
client = MongoClient(monog_url)
db = client.db

class Dao(object):
   def __init__(self):
      pass

   def save(self, user):
      db_doc = {
        'name': user.name,
        'email': user.email
      }
      db.users.save(db_doc)

   def getbyname(self, user):
      db_doc = {
        'name': user.name,
      }
      return db.users.find(db_doc)

To test this, I do not really want a test mongodb up and running! But also, I think I do not want to mock db.userssave and db.users.find because I want to actually be able to retrieve the data that I saved and make sure it is in the db. I think I need to create some fixtures per models that are in my memory and work with them. Just do I need an external tool to do so?

I am thinking of keeping some fake data like this, just do not know how to properly deal with it.

users = { 
    {'name' : 'Kelly', 'email' : 'kelly@gmail.com'},
    {'name': 'Sam', 'email': 'sam@gmail.com'}
}

Answer 3

I am using mock module for Python 2.7 to mock my other functions and using

unittest for writing unit tests.

I am wondering if mocking the MongoDB is different than using mock functionality ( mock.patch a function that is being called?) Or I need to use another different package for that purpose?

I do not think I want to have a test mongodb instance running. All I want is some tempo data and being able to call pymongo functionality. I am just a bit lost in thinking of is there a way to write a mock for a module (like pymongo ), or anything is achievable by mock module.

So appreciate if you could provide an example or tutorial on this.

Code to Test

from pymongo import MongoClient

monog_url = 'mongodb://localhost:27017'
client = MongoClient(monog_url)
db = client.db

class Dao(object):
   def __init__(self):
      pass

   def save(self, user):
      db_doc = {
        'name': user.name,
        'email': user.email
      }
      db.users.save(db_doc)

   def getbyname(self, user):
      db_doc = {
        'name': user.name,
      }
      return db.users.find(db_doc)

To test this, I do not really want a test mongodb up and running! But also, I think I do not want to mock db.userssave and db.users.find because I want to actually be able to retrieve the data that I saved and make sure it is in the db. I think I need to create some fixtures per models that are in my memory and work with them. Just do I need an external tool to do so?

I am thinking of keeping some fake data like this, just do not know how to properly deal with it.

users = { 
    {'name' : 'Kelly', 'email' : 'kelly@gmail.com'},
    {'name': 'Sam', 'email': 'sam@gmail.com'}
}

Answer 4

I am using mock module for Python 2.7 to mock my other functions and using

unittest for writing unit tests.

I am wondering if mocking the MongoDB is different than using mock functionality ( mock.patch a function that is being called?) Or I need to use another different package for that purpose?

I do not think I want to have a test mongodb instance running. All I want is some tempo data and being able to call pymongo functionality. I am just a bit lost in thinking of is there a way to write a mock for a module (like pymongo ), or anything is achievable by mock module.

So appreciate if you could provide an example or tutorial on this.

Code to Test

from pymongo import MongoClient

monog_url = 'mongodb://localhost:27017'
client = MongoClient(monog_url)
db = client.db

class Dao(object):
   def __init__(self):
      pass

   def save(self, user):
      db_doc = {
        'name': user.name,
        'email': user.email
      }
      db.users.save(db_doc)

   def getbyname(self, user):
      db_doc = {
        'name': user.name,
      }
      return db.users.find(db_doc)

To test this, I do not really want a test mongodb up and running! But also, I think I do not want to mock db.userssave and db.users.find because I want to actually be able to retrieve the data that I saved and make sure it is in the db. I think I need to create some fixtures per models that are in my memory and work with them. Just do I need an external tool to do so?

I am thinking of keeping some fake data like this, just do not know how to properly deal with it.

users = { 
    {'name' : 'Kelly', 'email' : 'kelly@gmail.com'},
    {'name': 'Sam', 'email': 'sam@gmail.com'}
}

Answer 5

I am using mock module for Python 2.7 to mock my other functions and using

unittest for writing unit tests.

I am wondering if mocking the MongoDB is different than using mock functionality ( mock.patch a function that is being called?) Or I need to use another different package for that purpose?

I do not think I want to have a test mongodb instance running. All I want is some tempo data and being able to call pymongo functionality. I am just a bit lost in thinking of is there a way to write a mock for a module (like pymongo ), or anything is achievable by mock module.

So appreciate if you could provide an example or tutorial on this.

Code to Test

from pymongo import MongoClient

monog_url = 'mongodb://localhost:27017'
client = MongoClient(monog_url)
db = client.db

class Dao(object):
   def __init__(self):
      pass

   def save(self, user):
      db_doc = {
        'name': user.name,
        'email': user.email
      }
      db.users.save(db_doc)

   def getbyname(self, user):
      db_doc = {
        'name': user.name,
      }
      return db.users.find(db_doc)

To test this, I do not really want a test mongodb up and running! But also, I think I do not want to mock db.userssave and db.users.find because I want to actually be able to retrieve the data that I saved and make sure it is in the db. I think I need to create some fixtures per models that are in my memory and work with them. Just do I need an external tool to do so?

I am thinking of keeping some fake data like this, just do not know how to properly deal with it.

users = { 
    {'name' : 'Kelly', 'email' : 'kelly@gmail.com'},
    {'name': 'Sam', 'email': 'sam@gmail.com'}
}

Answer 6

Mongomock is recommended way for testing, here's a runnable example to get started:

client.py

from dataclasses import dataclass

import pymongo

monog_url = 'mongodb://localhost:27018'
client = pymongo.MongoClient(monog_url)
db = client.db


@dataclass
class User:
    name: str
    email: str


class Dao:

    def save(self, db, user):
        db_doc = {
            'name': user.name,
            'email': user.email
        }
        return db.users.insert_one(db_doc).inserted_id

    def get_by_name(self, db, user):
        db_doc = {
            'name': user.name,
        }
        return db.users.find(db_doc)

test_client.py

import mongomock
import pytest

from client import Dao, User


class TestDao:

    @pytest.fixture
    def user(self):
        yield User(name='John', email='test@gmail.com')

    @pytest.fixture
    def dao(self):
        yield Dao()

    @pytest.fixture
    def client_mock(self):
        yield mongomock.MongoClient()

    @pytest.fixture
    def mock_db(self, client_mock):
        yield client_mock.db

    def test_save(self, mock_db, dao, user):
        id = dao.save(mock_db, user)

        users = list(mock_db.users.find())
        assert [obj for obj in users if obj['_id'] == id]
        assert len(users) == 1

    def test_get_by_name(self, mock_db, dao, user):
        dao.save(mock_db, user)
        found_user = next(dao.get_by_name(mock_db, user))
        found_user.pop('_id')
        assert found_user == user.__dict__

        unknown_user = User(name='Smith', email='john@gmail.com')
        found_user = next(dao.get_by_name(mock_db, unknown_user), None)
        assert found_user is None

Answer 7

Using @funnydman's example.

client.py

from pymongo import MongoClient


class MongoDB(object):
    def __init__(self) -> None:
        self.MONGO_URI ='mongodb://localhost:27018' 
        self.client = MongoClient(self.MONGO_URI)
        self.default_db = self.client.db

    def ingest_one(self, document: any, collection_name: str, db_name: str = None):

        if document:
            db_name = self.client[db_name] if db_name else self.default_db
            return db_name[collection_name].insert_one(document).inserted_id

    def find(self, query: dict, collection_name: str, db_name: str = None):
        db_name = self.client[db_name] if db_name else self.default_db
        return db_name[collection_name].find(query)

test_client.py

import mongomock
import pytest
import pytz
import datetime

from dataclasses import dataclass

from db.mongo_db import MongoDB


localtz = pytz.timezone('your_time_zone')


@dataclass
class Client:
    document: dict


class TestMongoDB:

    @pytest.fixture
    def client(self):
        yield Client(document={
            "name": "Juan Roman",
            "requestDate": str(datetime.datetime.now(localtz))
        })

    @pytest.fixture
    def mongo_db(self):
        yield MongoDB()

    @pytest.fixture
    def client_mock(self):
        yield mongomock.MongoClient()

    @pytest.fixture
    def mock_db(self, client_mock):
        yield client_mock.db

    def test_ingest_one(self, mock_db,  mongo_db, client):
        id_client = mongo_db.ingest_one(client.document, mock_db.collection.name, mock_db.name)
        stored_obj = mongo_db.find({'_id': id_client}, mock_db.collection.name, mock_db.name)

        assert [obj for obj in stored_obj if str(obj['_id']) == str(id_client)]