How to take out a list inside a dictionary value, and add newlines to it?

Question

How can I take out a list inside a dictionary value and make it part of the dictionary?

Here's the input that I'm given to work with:

[{'id': 1, 'step_and_result': [{'step': 'stepA', 'result': 'resultA'}, {'step': 'stepB', 'result': 'resultB'}, {'step': 'stepC', 'result': 'ResultC'}], 'other_key2': random_text}]

Here's the output that I am trying to get to:

[{'id': 1, 'step': 'stepA' + '\n' + 'stepB' + '\n' + 'stepC', 'result': 'resultA' + '\n' + 'resultB' + '\n' + 'resultC', 'other_key2': random_text}]

so that when I put the dictionary into a dataframe, the steps and results are shown on different lines, but within the same cell

I'm mostly stuck on how the step & result are give within a dictionary within a list within a dictionary within a list. Thanks for your help.

Answer 1

As Peter Leimbigler said, it's weird how result key becomes expected. Assuming you keep the same name, here's a solution using list comprehensions:

    # changed first 'result' key to 'expected'
    given_input = [{'id': 1, 'step_and_result': [{'step': 'stepA', 'expected': 'resultA'}, {'step': 'stepB', 'expected': 'resultB'}, {'step': 'stepC', 'expected': 'ResultC'}], 'other_key2': random_text}]

    given_input[0]['step'] = '\n'.join([d['step'] for d in given_input[0]['step_and_result']])
    given_input[0]['result'] = '\n'.join([d['expected'] for d in given_input[0]['step_and_result']])
    given_input[0].pop('step_and_result')

Answer 2

1st, i think you should make sure that all the objects have the same result key in the step_and_result. In your original example, stepA result is mapped to the "result" field, but in b and c it's mapped by "expected". is it possible to keep them all with the "result" key?

If so, here's a quick get-the-job done answer:

# this will be your converted end-result
converted = []

# we're going to iterator over each object and convert step objects into strings
for obj in original:
  # extract the step_and_result object 
  step_objs = obj['step_and_result']

  # we're going to store objects in list, and later we will join the list by our new-line delimeter once we're received all the results and steps
  results = []
  steps = []
  for s in step_objs:
    step, result = s['step'], s['result']
    steps.append(step)
    results.append(result)

  # add them to the end result my converting the lists into strings
  converted.append({
    'id': obj['id'],
    'step': '\n'.join(steps),
    'result': '\n'.join(results),
    'other_key2': obj['other_key2']
  })

Answer 3

If your keys inside step_and_result are all named result (not also expected ), and if you don't care what happens to other_key2 , here's a solution using json_normalize :

raw = [{'id': 1,
        'other_key2': 'asdf',
        'step_and_result': [{'result': 'resultA', 'step': 'stepA'},
                            {'result': 'resultB', 'step': 'stepB'},
                            {'result': 'ResultC', 'step': 'stepC'}]}]

from pandas.io.json import json_normalize
json_normalize(raw, record_path='step_and_result').sort_index(axis=1, ascending=False)

    step   result
0  stepA  resultA
1  stepB  resultB
2  stepC  ResultC

Answer 4

I have used a function so that the key can be either 'expected' or 'result'.

import pandas as pd
l=[{'id': 1,
    'step_and_result': [{'step': 'stepA', 'result': 'resultA'}, {'step': 'stepB', 'expected': 'resultB'}, {'step': 'stepC', 'expected': 'ResultC'}],
    'other_key2': 'random_text'}]
needed_l=l[0]['step_and_result']
def result_or_expected(d):
    if 'expected' in d.keys():
        return d['expected']
    return d['result']
new_dict_list={x['step']:result_or_expected(x) for x in needed_l}
df=pd.DataFrame(list(new_dict_list.items()), columns=['Step', 'Result'])
print(df.to_string(index=False))

Output

Step   Result
stepA  resultA
stepB  resultB
stepC  ResultC