How to create a Pandas DataFrame out of the keys from this JSON file?

Question

I have a csv file containing data in JSON structure and I want to create a dataframe out of it in a manner that all the keys become the column names and values the respective row values. Here's the csv file:

                                              events
0  {'id': 1245067280.0, 'eventId': 2, 'minute': 0
1  {'id': 1613373260.0, 'eventId': 2, 'minute': 0
2  {'id': 1560174527.0, 'eventId': 3, 'minute': 0
3  {'id': 1470954990.0, 'eventId': 4, 'minute': 0
4  {'id': 1628268979.0, 'eventId': 5, 'minute': 0

And here's what I want to create from this:

            id  eventId minute
0 1245067280.0  2        0
1 1613373260.0  2        0
2 1560174527.0  3        0
3 1470954990.0  4        0
4 1628268979.0  5        0

There are more key-value pairs but for the sake of brevity, I have only included three, id , eventId , and minute

What I've already tried is df.from_records() but that didn't change anything. Is there a Pandas/JSON function that allows this change easily or do people write code for this?

I've gone through many answers on here before posting and couldn't find anything specific to this. Most answers talk about creating a Dictionary from the JSON file and then converting to Series .

Answer 1

You can open the file and convert the rows into a list of dicts (parsing json strings) before dropping all into a DataFrame, eg.:

import csv
import json

with open('path_to_csv', 'r') as csvfile:
    reader = csv.reader(csvfile, delimiter=' ') # Assuming values in your file are separated with a space
    next(reader) # Skip header row
    records = [json.loads(row[1]) for row in reader] # Assuming that json strings are in the second column of the csv file

df = pd.DataFrame(records)

Answer 2

Hey you covert that file in json file and use json_normalize which is present in pandas.io.json. import json_normalize will be more efficient and compact.