熊猫：json多个文件，并以奇怪的格式连接

Question

I am having a hard time reading json files with a different structure than I am used to. The content of the json files are all inside brackets: [{content}].

This is what I normally do:

data_dir = 'data/filesDump'
filenames = os.listdir(data_dir)
filenames = [os.path.join(data_dir, f) for f in filenames if f.endswith('.json')]

train_df = pd.concat([pd.read_json(file, encoding='UTF-8') for file in filenames], 
           ignore_index = True)

I get this error:

ValueError: Expected object or value

The only thing different with the thousands json I got is that the content is in a bracket []. So I suspect this is giving json_read a problem? Anyone know how to load such format?

Sample (I may have made a mistake in brackets but that's just to give an idea):

[{"id":"value","title":"value","body":"text","categories":[{"id":value,"name":"name","keys":[{"id":value,"hits":["word1","word2"]},{"id":value,"hits":["word1","word2"]}],"date":value}]

Answer 1

Not all JSON files can be converted to a DataFrame, a specific format is required.

You should first convert your JSON files to Python structures with the standard json module, then you can modify the structure to fit the DataFrame constructor requirements.

For example, if your JSON has an extra bracket around the usual dictionary required to make a DataFrame, meaning the data is included in a list as sugested by @Atreus, you can remove it by taking only the first element of the list :

import json
struct=json.loads('[{"A":{"0":1,"1":2,"2":3},"B":{"0":4,"1":5,"2":6}}]')
print pd.DataFrame(struct[0])

outputs :

   A  B
0  1  4
1  2  5
2  3  6

Answer 2

因此，事实证明，我确实需要像manu所指的那样使用json.loads，但要注意以下几点：

json.load(open(file, encoding='utf-8-sig'))