如何将 Panda 类型的数据转换为 Panda.Dataframe？

Question

I have a object of which type is Panda and the print(object) is giving below output

            print(type(recomen_total))
            print(recomen_total)

Output is

<class 'pandas.core.frame.Pandas'>

Pandas(Index=12, instrument_1='XXXXXX', instrument_2='XXXX', trade_strategy='XXX', earliest_timestamp='2016-08-02T10:00:00+0530', latest_timestamp='2016-08-02T10:00:00+0530', xy_signal_count=1)

I want to convert this obejct in pd.DataFrame, how i can do it?

i tried pd.DataFrame(object), from_dict also, they are throwing error

Answer 1

Interestingly, it will not convert to a dataframe directly but to a series. Once this is converted to a series use the to_frame method of series to convert it to a DataFrame

import pandas as pd
df = pd.DataFrame({'col1': [1, 2], 'col2': [0.1, 0.2]},
                      index=['a', 'b'])

for row in df.itertuples():
    print(pd.Series(row).to_frame())

Hope this helps!!

EDIT

In case you want to save the column names use the _asdict() method like this:

import pandas as pd
df = pd.DataFrame({'col1': [1, 2], 'col2': [0.1, 0.2]},
                      index=['a', 'b'])

for row in df.itertuples():
    d = dict(row._asdict())
    print(pd.Series(d).to_frame())

Output:
         0
Index    a
col1     1
col2   0.1
         0
Index    b
col1     2
col2   0.2

Answer 2

To create new DataFrame from itertuples namedtuple you can use list() or Series too:

import pandas as pd

# source DataFrame
df = pd.DataFrame({'a': [1,2], 'b':[3,4]})
# empty DataFrame
df_new_fromAppend = pd.DataFrame(columns=['x','y'], data=None)

for r in df.itertuples():
    # create new DataFrame from itertuples() via list() ([1:] for skipping the index):
    df_new_fromList = pd.DataFrame([list(r)[1:]], columns=['c','d'])
    # or create new DataFrame from itertuples() via Series (drop(0) to remove index, T to transpose column to row) 
    df_new_fromSeries = pd.DataFrame(pd.Series(r).drop(0)).T
    # or use append() to insert row into existing DataFrame ([1:] for skipping the index):
    df_new_fromAppend.loc[df_new_fromAppend.shape[0]] = list(r)[1:]

print('df_new_fromList:')
print(df_new_fromList, '\n')
print('df_new_fromSeries:')
print(df_new_fromSeries, '\n')
print('df_new_fromAppend:')
print(df_new_fromAppend, '\n')

Output:

df_new_fromList:
   c  d
0  2  4 

df_new_fromSeries:
   1  2
0  2  4 

df_new_fromAppend:
   x  y
0  1  3
1  2  4 

To omit index, use param index=False (but I mostly need index for the iteration)

for r in df.itertuples(index=False):
    # the [1:] needn't be used, for example:
    df_new_fromAppend.loc[df_new_fromAppend.shape[0]] = list(r) 

Answer 3

The following works for me:

import pandas as pd
df = pd.DataFrame({'col1': [1, 2], 'col2': [0.1, 0.2]}, index=['a', 'b'])

for row in df.itertuples():    
   row_as_df = pd.DataFrame.from_records([row], columns=row._fields)
   print(row_as_df)

The result is:

Index  col1  col2
0     a     1   0.1
  Index  col1  col2
0     b     2   0.2

Sadly, AFAIU, there's no simple way to keep column names, without explicitly utilizing "protected attributes" such as _fields .

Answer 4

With some tweaks in @Igor's answer

I concluded with this satisfactory code which preserved column names and used as less of pandas code as possible.

import pandas as pd
df = pd.DataFrame({'col1': [1, 2], 'col2': [0.1, 0.2]})
# Or initialize another dataframe above

# Get list of column names
column_names = df.columns.values.tolist()

filtered_rows = []
for row in df.itertuples(index=False):
   # Some code logic to filter rows
   filtered_rows.append(row)

# Convert pandas.core.frame.Pandas to pandas.core.frame.Dataframe
# Combine filtered rows into a single dataframe
concatinated_df = pd.DataFrame.from_records(filtered_rows, columns=column_names)

concatinated_df.to_csv("path_to_csv", index=False)

The result is a csv containing:

col1  col2
   1   0.1
   2   0.2

Answer 5

To convert a list of objects returned by Pandas .itertuples to a DataFrame, while preserving the column names:

# Example source DF
data = [['cheetah', 120], ['human', 44.72], ['dragonfly', 54]]
source_df = pd.DataFrame(data, columns=['animal', 'top_speed'])

      animal  top_speed
0  cheetah       120.00
1  human          44.72
2  dragonfly      54.00

Since Pandas does not recommended building DataFrames by adding single rows in a for loop, we will iterate and build the DataFrame at the end:

WOW_THAT_IS_FAST = 50
list_ = list()
for animal in source_df.itertuples(index=False, name='animal'):
    if animal.top_speed > 50:
        list_.append(animal)

Now build the DF in a single command and without manually recreating the column names.

filtered_df = pd.DataFrame(list_)

      animal  top_speed
0  cheetah       120.00
2  dragonfly      54.00