Pandas Data Frame not Appending

Question

I am trying to append dataframes via for loop.

CODE

def redshift_to_pandas(sql_query,**kwargs):
# pass a sql query and return a pandas dataframe
 cur.execute(sql_query)
 columns_list = [desc[0] for desc in cur.description]
 data = pd.DataFrame(cur.fetchall(),columns=columns_list)
 return data

Input - all_schema = [('backup')]

Loop -

try:
 if len(all_schema) == 0:
  raise inputError("The Input has no schema selected. EXITING")
 else:
  modified_schemadf=pd.DataFrame(columns=['columns_name','status'])
  
  for i in range(len(all_schema)):
    #print (redshift_to_pandas("select '"+all_schema[i]+"' as columns_name,(select exists ( select distinct table_schema from information_schema.tables where table_schema like '%"+all_schema[i]+"')) as status",mechanism='append'))
    modified_schemadf.append(redshift_to_pandas("select '"+all_schema[i]+"' as columns_name,(select exists ( select distinct table_schema from information_schema.tables where table_schema like '%"+all_schema[i]+"')) as status",mechanism='append'))
    print (modified_schemadf)

except inputError as e:
   print(e.message)
   logger.error("UNEXPECTED INPUT FOUND, Please check the I/P List . EXITING")
  
print (modified_schemadf)

I feel the issue is very obvious but i dont seem to find the issue.

Here is the o/p -

So the the first print ( commented out ), does return me the correct result. the next steps ie appending the result to the declared dataframe ( name - modified_schemadf) is the problem area. When i print its value , it still throws a empty dataframe. For some reason the appending isnt happening.

When the code enters else , ie when the input is legit, there will be empty dataframe created called modified_schemadf. To this empty dataframe, there will be as many number of appends as there are inputs.

Thanks in Advance.

Please dont mind the indentations, copying might have affected them.

Answer 1

Isn't the issue just that you don't assign the appended dataframe? Try changing this line

modified_schemadf.append(redshift_to_pandas("select '"+all_schema[i]+"' as columns_name,(select exists ( select distinct table_schema from information_schema.tables where table_schema like '%"+all_schema[i]+"')) as status",mechanism='append'))

to this line

modified_schemadf = modified_schemadf.append(redshift_to_pandas("select '"+all_schema[i]+"' as columns_name,(select exists ( select distinct table_schema from information_schema.tables where table_schema like '%"+all_schema[i]+"')) as status",mechanism='append'))