从嵌套的JSON嵌套数组中获取键和值

Question

First time posting! I am converting JSON data (dictionary) from a server into a csv file. The keys and values taken are fine apart from the nest "Astronauts", which is an array. Basically every individual JSON string is a datum that may contains from 0 to an unlimited number of astronauts which features I would like to extract as independent values. For instance something like this:

• Astronaut1_Spaceships_First: Katabom
• Astronaut1_Spaceships_Second: The Kraken
• Astronaut1_name: Jebeddia
• (...)
• Astronaut2_gender: Hopefully female

and so on. The problem here is that the nest is set as an array and not a dictionary so I do not know what to do. I have tried the dpath library as well as flattering the nest but nothing did change. Any ideas?

import json
import os
import csv
import datetime
import dpath.util #Dpath library needs to be installed first

datum = {"Mission": "Make Earth Greater Again", "Objective": "Prove Earth is flat", "Astronauts": [{"Spaceships": {"First": "Katabom", "Second": "The Kraken"}, "Name": "Jebeddiah", "Gender": "Hopefully male", "Age": 35, "Prefered colleages": [], "Following missions": [{"Payment_status": "TO BE CONFIRMED"}]}, {"Spaceships": {"First": "The Kraken", "Second": "Minnus I"}, "Name": "Bob", "Gender": "Hopefully female", "Age": 23, "Prefered colleages": [], "Following missions": [{"Payment_status": "TO BE CONFIRMED"}]}]}

#Parsing process
        parsed = json.loads(datum)  #datum is the JSON string retrieved from the server

def flattenjson(parsed, delim):
    val = {}
    for i in parsed.keys():
        if isinstance(parsed[i], dict):
            get = flattenjson(parsed[i], delim)
            for j in get.keys():
                val[i + delim + j] = get[j]
        else:
        val[i] = parsed[i]

    return val
flattened = flattenjson(parsed,"__")

#process of creating csv file
keys=['Astronaut1_Spaceship_First','Astronaut2_Spaceship_Second', 'Astronaut1_Name]  #reduced to 3 keys for this example

 writer = csv.DictWriter(OD, keys ,restval='Null', delimiter=",", quotechar="\"", quoting=csv.QUOTE_ALL, dialect= "excel")
        writer.writerow(flattened)

.

#JSON DATA FROM SERVER
{
"Mission": "Make Earth Greater Again",
"Objective": "Prove Earth is flat",
"Astronauts": [    {
  "Spaceships": {
    "First": "Katabom",
    "Second": "The Kraken"
  },
  "Name": "Jebeddiah",
  "Gender": "Hopefully male",
  "Age": 35,
  "Prefered colleages": [],
  "Following missions": [
    {
      "Payment_status": "TO BE CONFIRMED"
    }
  ]
},
{
  "Spaceships": {
    "First": "The Kraken",
    "Second": "Minnus I"
  },
  "Name": "Bob",
  "Gender": "Hopefully female",
  "Age": 23,
  "Prefered colleages": [],
  "Following missions": [
    {
      "Payment_status": "TO BE CONFIRMED"
    }
  ]
},
  ]
}
]

Answer 1

Firstly, the datum you have defined here is not the datum that would be extracted from the server. The datum from the server would be a string. The datum you have in this program is already processed. Now, assuming datum to be:

datum = '{"Mission": "Make Earth Greater Again", "Objective": "Prove Earth is flat", "Astronauts": [{"Spaceships": {"First": "Katabom", "Second": "The Kraken"}, "Name": "Jebeddiah", "Gender": "Hopefully male", "Age": 35, "Prefered colleages": [], "Following missions": [{"Payment_status": "TO BE CONFIRMED"}]}, {"Spaceships": {"First": "The Kraken", "Second": "Minnus I"}, "Name": "Bob", "Gender": "Hopefully female", "Age": 23, "Prefered colleages": [], "Following missions": [{"Payment_status": "TO BE CONFIRMED"}]}]}'

You don't need the the dpath library. The problem here is that your json flattener doesn't handle embedded lists. Try using the one I've put below. Assuming that you want a one line csv file,

import json
def flattenjson(data, delim, topname=''):
    """JSON flattener that can handle embedded lists and dictionaries"""
    flattened = {}
    def internalflat(int_data, name=topname):
        if type(int_data) is dict:
            for key in int_data:
                internalflat(int_data[key], name + key + delim)
        elif type(int_data) is list:
            i = 1
            for elem in int_data:
                internalflat(elem, name + str(i) + delim)
                i += 1
        else:
            flattened[name[:-len(delim)]] = int_data
    internalflat(data)
    return flattened
#If you don't want mission or objective in csv file
flattened_astronauts = flattenjson(json.loads(datum)["Astronauts"], "__", "Astronaut")
keys = flattened_astronauts.keys().sort()
writer = csv.DictWriter(OD, keys ,restval='Null', delimiter=",", quotechar="\"", quoting=csv.QUOTE_ALL, dialect= "excel")
writer.writerow(flattened_astronauts)