I am trying to create a pandas dataframe from a collection of dictionaries that I read from a json file. The dictionaries are as follows -
d1 = {"DisplayName": "Test_drive", "permissions": {"read": True, "read_acp": True, "write": True, "write_acp": True}}
d2= {"DisplayName": "Log delivery","URI": "http://test_drive.com/Logs", "permissions": {"read": False, "read_acp": True, "write": True, "write_acp": False}}
I am trying to get these into a pandas dataframe. When I try to read them in a dataframe like following -
df = pd.DataFrame(d) **or** df = pd.DataFrame.from_dict(d)
it generates this -
DisplayName permissions
read Test_drive True
read_acp Test_drive True
write Test_drive True
write_acp Test_drive True
or read it as following -
df1 = pd.DataFrame(d).Transpose()
it generates this -
read read_acp write write_acp
DisplayName Test_drive Test_drive Test_drive Test_drive
permissions True True True True
I am trying to read these dictionaries and join them into one dataframe -
**DisplayName** **read** **read_acp** **write** **write_acp** URI
Test_drive True True True True NA
Log delivery False True True False http://test_drive.com/Logs
Is there any pytonic way to do this?
Create dataframe by appending and then reshape to the structure you need using pivot
df = pd.DataFrame.from_dict(d1).append(pd.DataFrame.from_dict(d2))
df.reset_index().pivot(index='DisplayName', columns='index', values='permissions')
To include URI
>>> df.reset_index().pivot(index='DisplayName', columns='index', values=['permissions', 'URI'])
permissions URI
index read read_acp write write_acp read read_acp write write_acp
DisplayName
Log delivery False True True False http://test_drive.com/Logs http://test_drive.com/Logs http://test_drive.com/Logs http://test_drive.com/Logs
Test_drive True True True True NaN NaN NaN NaN
import pandas as pd
# Input Data
d1 = {"DisplayName": "Test_drive", "permissions": {"read": True, "read_acp": True, "write": True, "write_acp": True}}
d2= {"DisplayName": "Log delivery","URI": "http://test_drive.com/Logs", "permissions": {"read": False, "read_acp": True, "write": True, "write_acp": False}}
# Convert to DataFrame
dicts = [d1, d2]
df_rows = [pd.DataFrame(d) for d in dicts]
df = pd.concat(df_rows, axis=0).reset_index(drop=False)
# Reshape As Desired
tp1 = df.pivot(index='DisplayName', columns='index', values='permissions')
answer = tp1.merge(df[['DisplayName', 'URI']].drop_duplicates(),
how='left',
left_index=True,
right_on='DisplayName').set_index('DisplayName')
Output:
>>> answer
read read_acp write write_acp URI
DisplayName
Log delivery False True True False http://test_drive.com/Logs
Test_drive True True True True NaN
Thank you Vishnudev and Max Power for your help. I think the following answer gives me the exact dataframe that I was trying to get.
d1 = {"DisplayName": "Test_drive", "permissions": {"read": True, "read_acp": True, "write": True, "write_acp": True}}
d2= {"DisplayName": "Log delivery","URI": "http://test_drive.com/Logs", "permissions": {"read": False, "read_acp": True, "write": True, "write_acp": False}}
df = pd.concat([pd.Series(d1),pd.Series(d2)], axis=1).transpose()
df = pd.concat([df.drop(['permissions'], axis=1),df['permissions'].apply(pd.Series)],axis=1)
**DisplayName URI read read_acp write write_acp**
0 Test_drive NaN True True True True
1 Log delivery http://test_drive.com/Logs False True True False
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.