[英]Python - append comma separated lists to a file
While making persistent API calls, I am looping over a large list in order to reorganize my data and save it to a file, like so: 在进行持久性API调用时,我遍历了一个较大的列表,以重新组织我的数据并将其保存到文件中,如下所示:
for item in music:
# initialize data container
data = defaultdict(list)
genre = item[0]
artist= item[1]
track= item[2]
# in actual code, api calls happen here, processing genre, artist and track
data['genre']= genre
data['artist'] = artist
data['track'] = track
# use 'a' -append mode
with open('data.json', mode='a') as f:
f.write(json.dumps([data], indent=4))
NOTE : Since I have a window of one hour to make api calls (after which token expires), I must save data to disk on the fly, inside the for loop
. 注意 :由于我有一个小时的窗口可以进行api调用(令牌将在此之后失效),因此必须在
for loop
内将数据动态保存到磁盘上。
The method above appends data to data.json
file, but my dumped lists are not comma separated and file ends up being populated like so: 上面的方法将数据追加到
data.json
文件,但是我的转储列表不是逗号分隔的,并且文件的最终填充方式如下:
[
{
"genre": "Alternative",
"artist": "Radiohead",
"album": "Ok computer"
}
]
[
{
"genre": "Eletronic",
"artist": "Kraftwerk",
"album": "Computer World"
}
]
So, how can I dump my data ending up with a list of lists separated by commas? 因此,如何转储以逗号分隔的列表结尾的数据?
One approach is to read the JSON file before writing to it. 一种方法是在写入JSON文件之前先读取它。
Ex: 例如:
import json
for item in music:
# initialize data container
data = defaultdict(list)
genre = item[0]
artist= item[1]
track= item[2]
data['genre']= genre
data['artist'] = artist
data['track'] = track
# Read JSON
with open('data.json', mode='r') as f:
fileData = json.load(f)
fileData.append(data)
with open('data.json', mode='w') as f:
f.write(json.dumps(fileData, indent=4))
Something like this would work 这样的事情会起作用
import json
music = [['Alternative', 'Radiohead', 'Ok computer'], ['Eletronic', 'Kraftwerk', 'Computer World']]
output = list()
for item in music:
data = dict()
genre = item[0]
artist= item[1]
track= item[2]
data['genre']= genre
data['artist'] = artist
data['track'] = track
output.append(data)
with open('data.json', mode='a') as f:
f.write(json.dumps(output, indent=4))
My data.json contains: 我的data.json包含:
[
{
"genre": "Alternative",
"track": "Ok computer",
"artist": "Radiohead"
},
{
"genre": "Eletronic",
"track": "Computer World",
"artist": "Kraftwerk"
}
]
For large datasets, pandas
(for serializing) and pickle
(for saving) work together like a charm. 对于大型数据集,
pandas
(用于序列化)和pickle
(用于保存)可以像一个吊饰一样协同工作。
df = pd.DataFrame()
for item in music:
# initialize data container
data = defaultdict(list)
genre = item[0]
artist= item[1]
track= item[2]
# in actual code, api calls happen here, processing genre, artist and track
data['genre']= genre
data['artist'] = artist
data['track'] = track
df = df.append(data, ignore_index=True)
df.to_pickle('data.pkl')
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.