使用 python 在 arangodb 中批量导入 .json 文件

Question

I have huge collection of.json files containing hundreds or thousands of documents I want to import to arangodb collections. Can I do it using python and if the answer is yes, can anyone send an example on how to do it from a list of files? ie:

for i in filelist:
    import i to collection

I have read the documentation but I couldn't find anything even resembling that

Answer 1

So after a lot of trial and error I found out that I had the answer in front of me. So I didn't need to import the.json file, I just needed to read it and then do a bulk import of documents. The code is like this:

a = db.collection('collection_name')
for x in list_of_json_files:
    with open(x,'r') as json_file:
        data = json.load(json_file)
        a.import_bulk(data)

So actually it was quite simple. In my implementation I am collecting the.json files from multiple folders and importing them to multiple collections. I am using the python-arango 5.4.0 driver

Answer 2

I had this same problem. Though your implementation will be slightly different, the answer you need (maybe not the one you're looking for) is to use the "bulk import" functionality.

Since ArangoDB doesn't have an "official" Python driver (that I know of), you will have to peruse other sources to give you a good idea on how to solve this.

• The HTTP bulk import/export docs provide curl commands, which can be neatly translated to Python web requests. Also see the section on headers and values .

• ArangoJS has a bulk import function, which works with an array of objects, so there's no special processing or preparation required.

I have also used the arangoimport tool to great effect. It's command-line, so it could be controlled from Python, or used stand-alone in a script. For me, the key here was making sure my data was in JSONL or "JSON Lines" format (each line of the file is a self-contained JSON object, no bounding array or comma separators).