使用 pymongo 查询大型 Mongodb 集合

Question

I want to query my mongodb collection which has more than 5k records, each record has key-value pair like

{
 "A" : "unique-value1",
 "B" : "service1",
 "C" : 1.2321,
 ...
},
...

here A will always have unique value, B has value like service1, service2, ....service8 and C is some float value.

what I want is to get a record like this with key-value pair.

{
 "A" : "unique-value1",
 "B" : "service1",
 "C" : 1.2321
}

{
  "A" : "unique-value2",
  "B" : "service2",
  "C" : 0.2321
}
{
  "A" : "unique-value3",
  "B" : "service1",
  "C" : 3.2321
}

I am not sure how to do this, earlier I used MapReduce but that time I was needed to generate records with A and C key value paire only but now since i also need B i do not know what should i do.

this is what i was doing

map_reduce = Code("""
        function () {
            emit(this.A, parseFloat(this.C));
        }
        """)
result = my_collection.map_reduce(map_reduce, reduce, out='temp_collection')

for doc in result.find({}):
    out = dict()
    out[doc['_id']] = doc['_id']
    out['cost'] = doc['value']
    out_handle.update_one(
        {'A': doc['_id']},
        {'$set': out},
        upsert=True
        )

Answer 1

Unless I've misunderstood what you need , it looks like you are making this harder than it need be. Just project the keys you want using the second parameter of the find method.

for record in db.testcollection.find({}, { 'A': 1, 'B': 1, 'C': 1}):
    db.existingornewcollection.replace_one({'_id': record['_id']}, record, upsert=True)

Full example:

from pymongo import MongoClient
from bson.json_util import dumps
db = MongoClient()['testdatabase']

db.testcollection.insert_one({
    "A": "unique-value1",
    "B": "service1",
    "C": 1.2321,
    "D": "D",
    "E": "E",
    "F": "F",
})

for record in db.testcollection.find({}, { 'A': 1, 'B': 1, 'C': 1}):
    db.existingornewcollection.replace_one({'_id': record['_id']}, record, upsert=True)

print(dumps(db.existingornewcollection.find_one({}, {'_id': 0}), indent=4))

gives:

{
    "A": "unique-value1",
    "B": "service1",
    "C": 1.2321
}