[英]Multiple GET Requests, merge json response into one array or json object
I am running one get request that returns some data in json format while also providing me the next url for the next page of data.我正在运行一个获取请求,它以 json 格式返回一些数据,同时还为我提供下一页数据的下一个 url。 I run a while loop grabbing all the data but want to append each new pages data to the existing object.我运行一个while循环来获取所有数据,但想将append每个新页面数据添加到现有的object。 Ultimately, I want one big json object or array.最终,我想要一个大的 json object 或阵列。
Here is my code so far, I do not think append is the right move here, as it's creating an index within the array for each page.到目前为止,这是我的代码,我不认为 append 是正确的举动,因为它在数组中为每个页面创建索引。 Rather I want one index or one json object with all the data combined.相反,我想要一个索引或一个 json object 与所有数据相结合。
host = 'https://xxxxx.com/api/v1/users'
headers = {'Accept': 'application/json', 'Content-Type': 'application/json',
'Authorization': 'xxxxx'}
response = requests.get('{}'.format(host),
headers = headers)
alist = []
keep_running = True
while keep_running:
json_response = response.json()
alist.append(json_response)
host = response.links['next']['url']
response = requests.get('{}'.format(host),
headers = headers)
keep_running = response.status_code == requests.codes.ok and 'next' in response.links.keys()
There are 200 objects per page and about 18 pages.每页有 200 个对象,大约 18 页。 I am getting an array of length 18 with 200 objects within each index.我得到一个长度为 18 的数组,每个索引中有 200 个对象。 Ideally, an array of length 18*200 = 3,600 would be want I'd want.理想情况下,我想要一个长度为 18*200 = 3,600 的数组。
I can achieve creating a dataframe/table using pandas however, I'd also like it in raw json.我可以使用 pandas 创建数据框/表,但是,我也希望在原始 json 中使用它。 Any ideas or help is appreciated.任何想法或帮助表示赞赏。
Assuming your json_response
is a list
, it looks like list.extend()
is what you're looking for.假设您的json_response
是一个list
,看起来list.extend()
就是您要查找的内容。 So instead of doing alist.append(json_response)
use alist.extend(json_response)
.因此,不要使用 alist.append(json_response alist.append(json_response)
使用alist.extend(json_response)
。
append
- adds an item to the end of the list. append
- 在列表末尾添加一个项目。
extend
- extends the list by appending all the items from the iterable. extend
- 通过附加可迭代的所有项目来扩展列表。
Example:例子:
x = [1, 2]
x.append([3, 4]) # gives [1, 2, [3, 4]]
x.extend([3, 4]) # gives [1, 2, 3, 4]
I'm not sure what you data actually looks like.我不确定您的数据实际上是什么样的。 But is sounds you can load the json-objects as a python-object.但是听起来你可以将 json-objects 作为 python-object 加载。 Append all of the data, and turn it back into a json-string afterwards. Append 所有数据,然后将其转回json字符串。
Like so:像这样:
import json
...
alist = []
keep_running = True
while keep_running:
json_response = response.json()
alist.append(json.loads(json_response))
host = response.links['next']['url']
response = requests.get('{}'.format(host),
headers = headers)
keep_running = response.status_code == requests.codes.ok and 'next' in response.links.keys()
json_string = json.dumps(alist)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.