How to transform Csv to Nested Json Using Pandas in Python

Question

I am having issues with converting csc to nested json. I read similar topics here, but unfortunately was not able to move later:( Thanks much for your advices. The simplified csv looks like:

And the desired json should be as follows:

{"data_type": 1,"person": {"id":1,"name": "Anne","surname": "Frank"},
{"data_type": 2,"address": {"id":1,"street": "de Bruijn","city": "Amsterdam"}
{"data_type": 1,"person": {"id":2,"name": "Terry","surname": "Gilliam"},
{"data_type": 2,"address": {"id":2,"street": "Westminster","city": "London"}

I am using pandas and I am struggling with following issues:

• don't know how to avoid inserting blank cells in structure (eg: no inserting of "Addres"" for data_type of 1
• sorting should be according to person id
• I can't achieve the desired structure - trying to use groupby, but still receiving something else...

Answer 1

Loop over each row and convert them according to your format:

cols = ['id', 'name', 'surname', 'street', 'city']

make_dict = lambda row: {
    'data_type': row['data_type'],
    row['data_name']: row[cols].dropna().to_dict()
}
df.apply(make_dict, axis=1).to_list()

Result:

[
  {
    "data_type": 1,
    "person": {"id": 1, "name": "Anne", "surname": "Frank"}
  },
  {
    "data_type": 2,
    "address": {"id": 1, "street": "de Brujin", "city": "Amsterdam"}
  },
  {
    "data_type": 1,
    "person": {"id": 2, "name": "Terry", "surname": "Gilliam"}
  },
  {
    "data_type": 2,
    "address": {"id": 2, "street": "Westminster", "city": "London"}
  }
]