How to append python pandas series columns to dataframe?

Question

I am a newbie on python pandas. I have a question on handling pandas dataframe. I use FRED (Federal Reserve Economic Data - St. Louis Fed) python api to get the big data. Below are the sources.

df_poverty = fred.search(search_str, order_by='title')  # returns pandas dataframe
mask_poverty = df_poverty.title == search_str  
df_poverty = df_poverty.loc[mask_poverty,['id']] 
if not df_poverty.empty:
   df_poverty_tmp = fred.get_series(df_poverty.iloc[0].id)  # returns another pandas dataframe
   print('******************')
   print(df_poverty.index)
   print('==================')
   print(df_poverty.head())
   print('==================')
   print(df_poverty_tmp.index)
   print('==================')
   print(df_poverty_tmp.head())

The above codes print the following results.

******************
Index(['PPAAAR05000A156NCEN'], dtype='object', name='series id')
==================
                                      id
series id                               
PPAAAR05000A156NCEN  PPAAAR05000A156NCEN
==================
DatetimeIndex(['1989-01-01', '1990-01-01', '1991-01-01', '1992-01-01',
               '1993-01-01', '1994-01-01', '1995-01-01', '1996-01-01',
               '1997-01-01', '1998-01-01', '1999-01-01', '2000-01-01',
               '2001-01-01', '2002-01-01', '2003-01-01', '2004-01-01',
               '2005-01-01', '2006-01-01', '2007-01-01', '2008-01-01',
               '2009-01-01', '2010-01-01', '2011-01-01', '2012-01-01',
               '2013-01-01', '2014-01-01', '2015-01-01', '2016-01-01',
               '2017-01-01', '2018-01-01'],
              dtype='datetime64[ns]', freq=None)
==================
1989-01-01    17.9
1990-01-01     NaN
1991-01-01     NaN
1992-01-01     NaN
1993-01-01    18.9
dtype: float64

My target format of results are feature matrix like below with time-series indexing,

1989-01-01     PPAAAR05000A156NCEN            17.9
1990-01-01     PPAAAR05000A156NCEN            NaN
1991-01-01     PPAAAR05000A156NCEN            NaN
1992-01-01     PPAAAR05000A156NCEN            NaN
1993-01-01     PPAAAR05000A156NCEN            18.9

I make python codes but the results are not satisfactory,

> df_poverty_tmp.append(df_poverty)

                        0                   id
1989-01-01 00:00:00  17.7                  NaN
1990-01-01 00:00:00   NaN                  NaN
1991-01-01 00:00:00   NaN                  NaN
1992-01-01 00:00:00   NaN                  NaN
1993-01-01 00:00:00  18.8                  NaN
1994-01-01 00:00:00   NaN                  NaN
1995-01-01 00:00:00  17.6                  NaN
1996-01-01 00:00:00  16.7                  NaN
1997-01-01 00:00:00  16.2                  NaN
1998-01-01 00:00:00  15.7                  NaN
PPAAAL01000A156NCEN   NaN  PPAAAL01000A156NCEN

I want to know how to insert the pandas series value into the middle of pandas dataframe columns. Any reply will be thankful.

== Update part

I add a few lines for your understanding.

df_poverty = df_poverty.loc[mask_poverty,['id', 'title', 'frequency_short', 'seasonal_adjustment_short']]
print(df_poverty)

The print line shows the following results

                                      id  title   frequency_short seasonal_adjustment_short
series id                                                           
PPAAAK02000A156NCEN  PPAAAK02000A156NCEN  test1           M                 NSA

Then the my expected feature is to be like below,

time                    id             title   frequency_short   value

1989-01-01     PPAAAR05000A156NCEN     test1         M            17.9
1990-01-01     PPAAAR05000A156NCEN     test1         M             NaN
1991-01-01     PPAAAR05000A156NCEN     test1         M             NaN
1992-01-01     PPAAAR05000A156NCEN     test1         M             NaN
1993-01-01     PPAAAR05000A156NCEN     test1         M             18.9

Answer 1

Since you have duplicate columns(two columns with the same content), I recommend you remove one.

df_poverty = df_poverty.drop("series id", 1)
df_poverty["time"] = DatetimeIndex.strftime(%Y-%m-%d)