How do I to convert from a list of list of lists to pandas DataFrame

Question

I have the following list of list of list and I needs to convert it into pandas DataFrame

[[array([['334', 0.8834404349327087, '1864004'],
         ['1613', 0.8664539456367493, '1864004'],
         ['1392', 0.6390842199325562, '1864004'],
         ['910', 0.6275900602340698, '1864004'],
         ['845', 0.6160776019096375, '1864004'],
         ['914', 0.4874180853366852, '1864004'],
         ['1109', 0.47274529933929443, '1864004'],
         ['215', 0.446095734834671, '1864004'],
         ['111', 0.4411593973636627, '1864004'],
         ['1303', 0.4356139302253723, '1864004']], dtype=object),
  array([['775', 1.040317416191101, '141243092'],
         ['1574', 0.9109242558479309, '141243092'],
         ['737', 0.8833655118942261, '141243092'],
         ['865', 0.7685815095901489, '141243092'],
         ['570', 0.7199063301086426, '141243092'],
         ['1704', 0.6063596606254578, '141243092'],
         ['604', 0.5963246822357178, '141243092'],
         ['389', 0.5376626253128052, '141243092'],
         ['815', 0.5313103795051575, '141243092'],
         ['543', 0.5247678756713867, '141243092']], dtype=object),
  array([['981', 0.8533183336257935, '1730822'],
         ['753', 0.7896251678466797, '1730822'],
         ['655', 0.664103090763092, '1730822'],
         ['345', 0.6045356392860413, '1730822'],
         ['588', 0.5033352375030518, '1730822'],
         ['496', 0.5005931258201599, '1730822'],
         ['809', 0.49608808755874634, '1730822'],
         ['884', 0.4614624083042145, '1730822'],
         ['876', 0.4543014466762543, '1730822'],
         ['1613', 0.44227200746536255, '1730822']], dtype=object)]]

With these values concatenated in the vertical.

I tried with the usually pd.DataFrame() but is not resulting since is a list of list of lists.

Regards

Answer 1

So I think what you want to do would look like the following:

from numpy import array
import pandas as pd

raw_data = [[array([['334', 0.8834404349327087, '1864004'],
         ['1613', 0.8664539456367493, '1864004'],
         ['1392', 0.6390842199325562, '1864004'],
         ['910', 0.6275900602340698, '1864004'],
         ['845', 0.6160776019096375, '1864004'],
         ['914', 0.4874180853366852, '1864004'],
         ['1109', 0.47274529933929443, '1864004'],
         ['215', 0.446095734834671, '1864004'],
         ['111', 0.4411593973636627, '1864004'],
         ['1303', 0.4356139302253723, '1864004']], dtype=object),
  array([['775', 1.040317416191101, '141243092'],
         ['1574', 0.9109242558479309, '141243092'],
         ['737', 0.8833655118942261, '141243092'],
         ['865', 0.7685815095901489, '141243092'],
         ['570', 0.7199063301086426, '141243092'],
         ['1704', 0.6063596606254578, '141243092'],
         ['604', 0.5963246822357178, '141243092'],
         ['389', 0.5376626253128052, '141243092'],
         ['815', 0.5313103795051575, '141243092'],
         ['543', 0.5247678756713867, '141243092']], dtype=object),
  array([['981', 0.8533183336257935, '1730822'],
         ['753', 0.7896251678466797, '1730822'],
         ['655', 0.664103090763092, '1730822'],
         ['345', 0.6045356392860413, '1730822'],
         ['588', 0.5033352375030518, '1730822'],
         ['496', 0.5005931258201599, '1730822'],
         ['809', 0.49608808755874634, '1730822'],
         ['884', 0.4614624083042145, '1730822'],
         ['876', 0.4543014466762543, '1730822'],
         ['1613', 0.44227200746536255, '1730822']], dtype=object)]]

data = raw_data[0]  # raw data is a list with one element for some reason
pd.concat(pd.DataFrame(datum) for datum in data)

Answer 2

Use np.vstack to get them all together from first element of your list. Then just simply convert it to pandas:

df = pd.DataFrame(data=np.vstack(list[0]), columns=['col1', 'col2', 'col3'])