Appending to an empty data frame in Pandas using a for loop

Question

I am writing because I am having an issue with a for loop which fills a dataframe when it is empty. Unfortunately, the posts Filling empty python dataframe using loops , Appending to an empty data frame in Pandas? , Creating an empty Pandas DataFrame, then filling it? did not help me to solve it.

My attempt aims, first, at finding the empty dataframes in the list "listDataframe" and then, wants to fill them with some chosen columns. I believe my code is clearer than my explanation. What I can't do is to save the new dataframe using its original name. Here my attempt:

for k,j in zip(listOwner,listDataframe):
for y in j:
    if y.empty:
        data = pd.DataFrame({"Event Date": list_test_2, "Site Group Name" : k, "Impressions" : 0})
        y = pd.concat([data,y])
        #y = y.append(data)

where "listOwner", "listDataframe" and "list_test_2" are, respectively, given by:

listOwner = ['OWNER ONE', 'OWNER TWO', 'OWNER THREE', 'OWNER FOUR']
listDataframe = [df_a,df_b,df_c,df_d]

with

df_a = [df_ap_1, df_di_1, df_er_diret_1, df_er_s_1]
df_b = [df_ap_2, df_di_2, df_er_diret_2, df_er_s_2]
df_c = [df_ap_3, df_di_3, df_er_diret_3, df_er_s_3]
df_d = [df_ap_4, df_di_4, df_er_diret_4, df_er_s_4]

and

list_test_2 = []
for i in range(1,8):
    f = (datetime.today() - timedelta(days=i)).date()
    list_test_2.append(datetime.combine(f, datetime.min.time()))

The empty dataframe were df_ap_1 and df_ap_3. After running the above lines (using both concat and append) if I call these two dataframes they are still empty. Any idea why that happens and how to overcome this issue?

UPDATE

In order to avoid both append and concat, I tried to use the coming attempt (again with no success).

for k,j in zip(listOwner,listDataframe):
    for y in j:
        if y.empty:
            y = pd.DataFrame({"Event Date": list_test_2, "Site Group Name" : k, "Impressions" : 0})

The two desired result should be:

where the first dataframe should be called df_ap_1 while the second one df_ap_3 .

Thanks in advance.

Drigo

Answer 1

Here's a way to do it:

import pandas as pd

columns = ['Event Date', 'Site Group Name', 'Impressions']
df_ap_1 = pd.DataFrame(columns=columns) #empty dataframe
df_di_1 = pd.DataFrame(columns=columns) #empty dataframe
df_ap_2 = pd.DataFrame({'Event Date':[1], 'Site Group Name':[2], 'Impressions': [3]}) #non-empty dataframe
df_di_2 = pd.DataFrame(columns=columns) #empty dataframe

df_a = [df_ap_1, df_di_1]
df_b = [df_ap_2, df_di_2]
listDataframe = [df_a,df_b]

list_test_2 = 'foo'
listOwner = ['OWNER ONE', 'OWNER TWO']

def appendOwner(df, owner, list_test_2):
    #appends a row to a dataframe for each row in listOwner
    new_row = {'Event Date': list_test_2,
               'Site Group Name': owner,
               'Impressions': 0,
               }
    df.loc[len(df)] = new_row

for owner, dfList in zip(listOwner, listDataframe):
    for df in dfList:
        if df.empty:
            appendOwner(df, owner, list_test_2)

print(listDataframe)

You can use the appendOwner function to append the rows from listOwner to an empty dataframe.