如何在文件夹结构中读取多个 json 文件？

Question

I'm trying to read Training data for machine learning in json files, but they are stored in nested folders.

I'd like to know how can I read each json file into pandas frame.

Answer 1

Imagine there are three JSON files in a nested folder under data folder.

$ tree data
data
├── date1
│   ├── date2
│   │   └── file1_date2.json
│   └── file1.json
└── file1.json

glob2 module can be used to fetch the JSON files recursively. glob returns a list of files.

from glob2 import glob
jsonFiles = glob('data/**/*.json') #Can be used absolute or relative paths
print(jsonFiles)

['data/file1.json',  'data/date1/file1.json',  'data/date1/date2/file1_date2.json']

JSON files ban be loaded into dataframe by iterating thru the list jsonFiles .

dfList = []
for jsonFile in jsonFiles:
    df = pd.read_json(jsonFile)
    dfList.append(df)
    
dfTrainingDF = pd.concat(dfList, axis=0)