Converting API output from a dictionary to a dataframe (Python)

Question

I have fed some data into a TravelTime ( https://github.com/traveltime-dev/traveltime-python-sdk ) API which calculates the time it takes to drive between 2 locations. The result of this (called out) is a dictionary with that looks like this:

{'results': [{'search_id': 'arrival_one_to_many',
   'locations': [{'id': 'KA8 0EU', 'properties': {'travel_time': 2646}},
    {'id': 'KA21 5DT', 'properties': {'travel_time': 392}}],
   'unreachable': []}]}

However, I need a table that would look a bit like this:

search_id	id	Travel Time
arrival_one_to_many	KA21 5DT	2646
arrival_one_to_many	KA21 5DT	392

I've tried converting this dictionary to a dataframe using out_2 = pd.DataFrame.from_dict(out)

This shows as a dataframe with one column called results, so I tried use out_2['results'].str.split(',', expand=True) to split this into multiple columns at the comma delimiters but got an error:

	0
0	NaN

Is anyone able to help me to get this dictionary to a readable and useable dataframe/table?

Thanks

Answer 1

First, this json to be parsed to fetch required value. Once those values are fetched, then we can store them into dataframe.

Below is the code to parse this json (PS: I have saved json in one file) and these values added to DataFrame.

import json
import pandas as pd

f = open('Json_file.json')
req_file = json.load(f)
df = pd.DataFrame()
for i in req_file['results']:
    dict_new = dict()
    dict_new['searchid'] = i['search_id']
    for j in i['locations']:
        dict_new['location_id'] = j['id']
        dict_new['travel_time'] = j['properties']['travel_time']
        df = df.append(dict_new, ignore_index=True)
        
print(df)

Below is the output of above code:

              searchid location_id  travel_time
0  arrival_one_to_many     KA8 0EU       2646.0
1  arrival_one_to_many    KA21 5DT        392.0

Answer 2

@MICHAELKM22 since you are not using all the keys from the dictionary you wont be able to convert it directly to dataframe. First extract required keys and then convert it into dataframe.

df_list = []
for res in data['results']:
    serch_id = res['search_id']
    for loc in res['locations']:
        temp_df = {}
        temp_df['search_id'] = res['search_id']
        temp_df['id'] = loc["id"]
        temp_df['travel_time'] = loc["properties"]['travel_time']
        
        df_list.append(temp_df)
df = pd.DataFrame(df_list)

    search_id           id          travel_time
0   arrival_one_to_many KA8 0EU     2646
1   arrival_one_to_many KA21 5DT    392