How do I load a json file into mongoDB?

Question

I have a json file with some information in this format:

{ "_id" : ObjectId("xxx"), "date_time" : ISODate("2014-06-11T19:16:45Z"), "name" : "AAA", "phone_no" : "111", "address" : "BBB", "categories" : "CCC" }  
{ "_id" : ObjectId("yyy"), "date_time" : ISODate("2014-06-11T19:16:44Z"), "name" : "EEE", "phone_no" : "222", "address" : "FFF", "categories" : "GGG" }  
{ "_id" : ObjectId("zzz"), "date_time" : ISODate("2014-06-11T19:16:46Z"), "name" : "HHH", "phone_no" : "333", "address" : "III", "categories" : "JJJ" }

The code I'm using is this:

db = pymongo.MongoClient().test  
path ='/home/files'  
for f in listdir(path):  
    filepath = path+'/'+f  
    data = []     
    for line in open(filepath):  
        try:  
            data.append(json.loads(line))  
        except:  
            pass  
    db.temp.insert(data)  

This results in an error stating empty bulk write is not possible. Basically, the json.loads(line) never works. Is it the format of the json file that's the issue? Should the variable 'data' be declared some other way?

How do I load this file into mongoDB ?

Answer 1

json doesnt know what an ObjectID is or an ISODate ... it can only handle simple types... you could try and load the data with yaml if you have defined serialization rules for those clases ... or you can just use simple strings in the line.

for line in open(filepath):
    line = re.sub("[a-zA-Z_]+\(([^)]+)\)","\\1",line)
    print json.loads(line)
    ... #do your thing

this will remove the class calls converting

{ "_id" : ObjectId("xxx"), "date_time" : ISODate("2014-06-11T19:16:45Z"), "name" : "AAA", "phone_no" : "111", "address" : "BBB", "categories" : "CCC" }  

to

{ "_id":"xxx", "date_time" : "2014-06-11T19:16:45Z", "name" : "AAA", "phone_no" : "111", "address" : "BBB", "categories" : "CCC" }  

which you should then be able to load with json