[英]Python append json to json file in a while loop
I'm trying to get all users information from GitHub API using Python Requests library. 我正在尝试使用Python Requests库从GitHub API获取所有用户信息。 Here is my code:
这是我的代码:
import requests
import json
url = 'https://api.github.com/users'
token = "my_token"
headers = {'Authorization': 'token %s' % token}
r = requests.get(url, headers=headers)
users = r.json()
with open('users.json', 'w') as outfile:
json.dump(users, outfile)
I can dump first page of users into a json file by now. 现在,我可以将用户的第一页转储到json文件中。 I can also find the 'next' page's url:
我还可以找到“下一个”页面的网址:
next_url = r.links['next'].get('url')
r2 = requests.get(next_url, headers=headers)
users2 = r2.json()
Since I don't know how many pages yet, how can I append 2nd, 3rd... page to 'users.json' sequentially in a while loop as fast as possible? 由于我还不知道有多少页,如何在一个while循环中尽可能快地将第二,第三...页依次追加到“ users.json”?
Thanks! 谢谢!
Append the data you get from the requests
query to a list and move on to the next query. 将您从
requests
查询中获得的数据追加到列表中,然后继续进行下一个查询。
Once you have all of the data you want, then proceed to try to concatenate the data into a file or into an object. 拥有所需的所有数据后,然后继续尝试将数据串联到文件或对象中。 You can also use
threading
to do multiple queries in parallel, but most likely there is going to be rate limiting on the api. 您还可以使用
threading
并行执行多个查询,但是最有可能在api上进行速率限制。
First, you need to open file in 'a' mode, otherwise subsequence write will overwrite everything 首先,您需要以“ a”模式打开文件,否则子序列写入将覆盖所有内容
import requests
import json
url = 'https://api.github.com/users'
token = "my_token"
headers = {'Authorization': 'token %s' % token}
outfile = open('users.json', 'a')
while True:
r = requests.get(url, headers=headers)
users = r.json()
json.dump(users, outfile)
url = r.links['next'].get('url')
# I don't know what Github return in case there is no more users, so you need to double check by yourself
if url == '':
break
outfile.close()
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.