How can I create a pandas data frame in a certain way?

Question

I need to create a pandas dataframe that contains all of the required information where each row of the dataframe should be one track. I also need to sort the dataframe by popularity score, so that the most popular track is at the top and the least popular is at the bottom. I tried many ways but they did not work. Your help is much appreciated.

I am sharing my nested dictionary.

{'Artist name': ['Paramore', 'Weezer', 'Lizzo'],
 'Track name': (['Still into You',
   "Ain't It Fun",
   'Hard Times',
   'Misery Business',
   'The Only Exception',
   'Ignorance',
   'Rose-Colored Boy',
   'Fake Happy',
   "That's What You Get",
   'Brick by Boring Brick'],
  ['Island In The Sun',
   "Say It Ain't So",
   'Buddy Holly',
   'Beverly Hills',
   'Africa',
   'The End of the Game',
   'Hash Pipe',
   'Undone - The Sweater Song',
   'My Name Is Jonas',
   'Take On Me'],
  ['Truth Hurts',
   'Good As Hell',
   'Good As Hell (feat. Ariana Grande) - Remix',
   'Juice',
   'Boys',
   'Tempo (feat. Missy Elliott)',
   'Blame It on Your Love (feat. Lizzo)',
   'Soulmate',
   'Water Me',
   'Like A Girl']),
 'Release date': (['2013-04-05',
   '2013-04-05',
   '2017-05-12',
   '2007-06-11',
   '2009-09-28',
   '2009-09-28',
   '2017-05-12',
   '2017-05-12',
   '2007-06-11',
   '2009-09-28'],
  ['2001-05-15',
   '1994-05-10',
   '1994-05-10',
   '2005-05-10',
   '2019-01-24',
   '2019-09-10',
   '2001-05-15',
   '1994-05-10',
   '1994-05-10',
   '2019-01-24'],
  ['2019-05-03',
   '2016-03-09',
   '2019-10-25',
   '2019-04-19',
   '2019-04-18',
   '2019-04-19',
   '2019-09-13',
   '2019-04-19',
   '2019-04-18',
   '2019-04-19']),
 'Popularity score': ([76, 74, 73, 73, 72, 69, 66, 66, 65, 65],
  [77, 75, 73, 71, 67, 67, 66, 65, 63, 62],
  [94, 90, 86, 84, 72, 78, 68, 72, 58, 71])}

Answer 1

There are definitely more efficient ways, but here's a solution

import pandas as pd

def gen_artist_frame(d):
    categories = [c for c in d.keys()]

    for idx, artist in enumerate(d['Artist name']):

        artist_mat = [d[j][idx] for j in categories[1:]]

        artist_frame = pd.DataFrame(artist_mat, index=categories[1:]).T

        artist_frame[categories[0]] = artist

        yield artist_frame

def collapse_nested_artist(d):
    return pd.concat([
        a for a in gen_artist_frame(d)
        ])

d = {'Artist name': ['Paramore', 'Weezer', 'Lizzo'],
 'Track name': (['Still into You',
   "Ain't It Fun",
   'Hard Times',
   'Misery Business',
   'The Only Exception',
   'Ignorance',
   'Rose-Colored Boy',
   'Fake Happy',
   "That's What You Get",
   'Brick by Boring Brick'],
  ['Island In The Sun',
   "Say It Ain't So",
   'Buddy Holly',
   'Beverly Hills',
   'Africa',
   'The End of the Game',
   'Hash Pipe',
   'Undone - The Sweater Song',
   'My Name Is Jonas',
   'Take On Me'],
  ['Truth Hurts',
   'Good As Hell',
   'Good As Hell (feat. Ariana Grande) - Remix',
   'Juice',
   'Boys',
   'Tempo (feat. Missy Elliott)',
   'Blame It on Your Love (feat. Lizzo)',
   'Soulmate',
   'Water Me',
   'Like A Girl']),
 'Release date': (['2013-04-05',
   '2013-04-05',
   '2017-05-12',
   '2007-06-11',
   '2009-09-28',
   '2009-09-28',
   '2017-05-12',
   '2017-05-12',
   '2007-06-11',
   '2009-09-28'],
  ['2001-05-15',
   '1994-05-10',
   '1994-05-10',
   '2005-05-10',
   '2019-01-24',
   '2019-09-10',
   '2001-05-15',
   '1994-05-10',
   '1994-05-10',
   '2019-01-24'],
  ['2019-05-03',
   '2016-03-09',
   '2019-10-25',
   '2019-04-19',
   '2019-04-18',
   '2019-04-19',
   '2019-09-13',
   '2019-04-19',
   '2019-04-18',
   '2019-04-19']),
 'Popularity score': ([76, 74, 73, 73, 72, 69, 66, 66, 65, 65],
  [77, 75, 73, 71, 67, 67, 66, 65, 63, 62],
  [94, 90, 86, 84, 72, 78, 68, 72, 58, 71])}

frame = collapse_nested_artist(d)

Answer 2

Dictionaries as dataframes are easier to handle if all the values in the key value pairings are the same size, and can make it more straightforward. If possible, I would reformat your dictionary slightly. For example, nest each column into the artist to avoid assumptions about positions:

ex = {'foo':{'title':[1,2],'letter':['a','b']},
      'bar':{'title':[3,4],'letter':['c','d']}, 
      'fob':{'title':[5,6],'letter':['e','f']},
     }

df = []
for key, value in ex.items():
    minidf = pd.DataFrame(value)
    minidf['label'] = key
    df.append(minidf)
pd.concat(df, ignore_index=True)

will return

   title letter label
0      1      a   foo
1      2      b   foo
2      3      c   bar
3      4      d   bar
4      5      e   fob
5      6      f   fob