用嵌套的dict键值对创建包含项的列表

Question

I would like to create a new list with items from a large nested dict.

Here is a snippet of the nested dict:

AcceptedAnswersPython_combined.json

{
  "items": [
    {
      "answers": [
        {
          "creation_date": 1533083368,
          "is_accepted": false
        },
        {
          "creation_date": 1533083567,
          "is_accepted": false
        },
        {
          "creation_date": 1533083754,
          "is_accepted": true
        },
        {
          "creation_date": 1533084669,
          "is_accepted": false
        },
        {
          "creation_date": 1533089107,
          "is_accepted": false
        }
      ],
      "creation_date": 1533083248,
      "tags": [
        "python",
        "pandas",
        "dataframe"
      ]
    },
    {
      "answers": [
        {
          "creation_date": 1533084137,
          "is_accepted": true
        }
      ],
      "creation_date": 1533083367,
      "tags": [
        "python",
        "binary-search-tree"
      ]
    }
  ]
} 

The new list should contain the creation_date of each item as many times as there are dicts inside the answers list. So in case of the code snippet above the new list should look like this:

question_date_per_answer = [[1533083248, 1533083248, 1533083248 , 1533083248, 1533083248], [1533083367]]

The reason why I need this new list is that I would like to determine the difference between each answers creation_date and its associated question creation_date (stated inside the each items dict).

This new list should look like this in pandas Dataframe:

     question creation date answer creation date  
0          1533083248             1533083368               
1          1533083248             1533083567               
2          1533083248             1533083754                
3          1533083248             1533084669               
4          1533083248             1533089107               
5          1533083367             1533084137

I can iterate through all question like so:

items = json.load(open('AcceptedAnswersPython_combined.json'))['items']
question_creation_date = [item['creation_date'] for item in items]

But this leaves me with a list which is unequal to the number of answers creation_date .

I can't get my head around this.
So how do I create such a list where the amount of question creation dates is equal to the amount of answer creation dates? (like question_date_per_answer )

Thanks in advance.

Answer 1

you need to iterate over item["answers"] and then get creation_date for each answer in oreder to get answer creation dates.

my_json = """{
"items": [
    {
    "answers": [
        {
        "creation_date": 1533083368,
        "is_accepted": false
        },
        {
        "creation_date": 1533083567,
        "is_accepted": false
        },
        {
        "creation_date": 1533083754,
        "is_accepted": true
        },
        {
        "creation_date": 1533084669,
        "is_accepted": false
        },
        {
        "creation_date": 1533089107,
        "is_accepted": false
        }
    ],
    "creation_date": 1533083248,
    "tags": [
        "python",
        "pandas",
        "dataframe"
    ]
    },
    {
    "answers": [
        {
        "creation_date": 1533084137,
        "is_accepted": true
        }
    ],
    "creation_date": 1533083367,
    "tags": [
        "python",
        "binary-search-tree"
    ]
    }
]
}"""

import json

data = json.loads(my_json)
dates = [(question["creation_date"], answer["creation_date"])
         for question in data["items"] for answer in question["answers"]]
print(dates)

Answer 2

You can still work with the list at hand.
Lets try making a dataframe from the list that you already have-

l = [[1533083248, 1533083248, 1533083248 , 1533083248, 1533083248], [1533083367]]
df = pd.DataFrame(l)

Unfortunately you get the following-

0   1   2   3   4
0   1533083248  1.533083e+09    1.533083e+09    1.533083e+09    1.533083e+09
1   1533083367  NaN     NaN     NaN     NaN

So we need to transpose it. For that lets do the following -

from itertools import zip_longest
k = list(list(zip_longest(*l))) #Unless the list will be truncated to the length of shortest list.
df = pd.DataFrame(k)

Output-

0   1
0   1533083248  1.533083e+09
1   1533083248  NaN
2   1533083248  NaN
3   1533083248  NaN
4   1533083248  NaN

Now we will forward fill the NaNs with the previous value by - df.fillna(method='ffill')
Whole snippet -

from itertools import zip_longest
l=[1533083248, 1533083248, 1533083248 , 1533083248, 1533083248], [1533083367]
k=list(list(zip_longest(*l)))
df = pd.DataFrame(k)
df.fillna(method='ffill')

Voila -

    0   1
0   1533083248  1.533083e+09
1   1533083248  1.533083e+09
2   1533083248  1.533083e+09
3   1533083248  1.533083e+09
4   1533083248  1.533083e+09