Python how to open multiple json files in a directory

Question

I am trying to open a directory in which there are multiple json files, to then make a data frame of the data in each of them, I try this:

    for file in os.listdir('Datasets/'):
        json_read = pd.read_json(file)

However, it gives me an error: ValueError: Expected object or value

As I inspect the type of the files, it says they are class str. When opening a single file in the directory with read_json it does work correctly as the file is recognized as json. I am not quite sure why the files are turned into strings nor how to solve it. Do you have any tips?

Thanks in advance!

Answer 1

import os
import pandas as pd
base_dir = '/path/to/dir'

#Get all files in the directory

data_list = []
for file in os.listdir(base_dir):

    #If file is a json, construct it's full path and open it, append all json data to list
    if file.endswith('json'):
        json_path = os.path.join(base_dir, file)
        json_data = pd.read_json(json_path, lines=True)
        data_list.append(json_data)

print(data_list)

Answer 2

You probably need to build a list of DataFrames. You may not be able to process every file in the given directory so try this:

import pandas as pd
from glob import glob
from os.path import join
BASEDIR = 'Datasets'
dataframes = []
for file in glob(join(BASEDIR, '*.json')):
    try:
        dataframes.append(pd.read_json(file))
    except ValueError:
        print(f'Unable to process {file}')
print(f'Successfully constructed {len(dataframes)} dataframes')

Answer 3

import os

import json
#heres some information about get list of file in a folder: <https://www.geeksforgeeks.org/python-list-files-in-a-directory/>
#heres some information about how to open a json file: <https://www.geeksforgeeks.org/read-json-file-using-python/>
path = "./data"
file_list = os.listdir(path) #opens the directory form path and gets name of file

for i in range(len(file_list)): #loop the list with index number
  
  current = open(path+"/"+file_list[i]) #opens the file in order by current index number
  data = json.load(current) #loads the data form file

  for k in data['01']: # 
      print(k)

output

main.json :
{'name': 'Nava', 'Surname': 'No need for to that'}
data.json :
{'name': 'Nava', 'watchs': 'Anime'}

heres a link to run code online https://replit.com/@Nava10Y/opening-open-multiple-json-files-in-a-directory