For 循环中递增 Append 到 JSON 文件

Question

Is there a way to append single JSON objects to a json file while in a for loop in python. I would prefer not store all my data in one giant json object and dump it all at once, as I am planning on performing millions of API requests. I would like to make a single API request, dump the result into a JSON file and then move to the next API request and dump that into the same JSON file.

The below code overwrites the JSON file, I am looking for something that appends.

for url in urls:
    r = sesh.get(url)
    data = r.json()

    with open('data.json', 'w') as outfile:
        json.dump(data, outfile)

Such that:

with open('data.json') as outfile:
    data = json.load(data, outfile)

type(data)
>> dict

r.json looks something like this:

{'attribute1':1, 'attribute2':10}

Answer 1

Update

Well since I don't have access to your API I just placed some sample responses, in the format you supplied, inside an array.

import json

urls = ['{"attribute1":1, "attribute2":10}', '{"attribute1":67, "attribute2":32}', '{"attribute1":37, "attribute2":12}'];
json_arr = []

for url in urls:
    data = json.loads(url)
    json_arr.append(data)
    with open('data.json', 'w') as outfile:
        json.dump(json_arr, outfile)

Basically we keep an array and append each API response to that array. Then, we can write the accumulative JSON to a file. Also if you want to update the same JSON file on different executions of the code, you can just read the existing output file into an array, in the beginning of the code, and then carry on with my example.

---

---

Change write mode to append

Try changing this:

with open('data.json', 'a') as outfile:

To this:

 with open('data.json', 'a') as outfile: 

Answer 2

The previous answer is surprisingly close to what you need to do. So I will build upon it.

import json

json_arr = ['{"attribute1":1, "attribute2":10}', '{"attribute1":67, "attribute2":32}', '{"attribute1":37, "attribute2":12}'];

with open('data.json', 'w') as outfile:
  outfile.write('[')

for element in json_arr:
    with open('data.json', 'w') as outfile:
        json.dump(element, outfile)
        outfile.write(',')
with open('data.json', 'a') as outfile:
  outfile.write(']')

Answer 3

Was this ^ problem ever solved? Having a similar issues where after running a loop, my json becomes invalid. I am pulling out all of the objects in a for loop where a key matches a certain name. if that matches, I append it to a list and then at the end I am calling json.dumps on that list. but I am losing the proper json format to keep the files valid. Any help would be amazing! Example: The top 2 json objects were pulled from one file, and the bottom two were pulled from another one. but the format is now shot.

 [ { "item_name": "Dusty Rhodes", "player_name": "Dusty Rhodes", "team": null, "year": null, "images": [ { "url": "", "path": "" } ] }, { "item_name": "Dusty Rhodes", "player_name": "Dusty Rhodes", "team": null, "year": null, "images": [ { "url": "", "path": "" ] } ][ { "item_name": "Dusty Rhodes", "player_name": "Dusty Rhodes", "team": "", "year": null, "images": [] } ]