In python pandas, how can I re-sample and interpolate a DataFrame?

Question

I have a pd DataFrame, typically on this format:

   1       2          3          4  
0.1100 0.0000E+00 1.0000E+00 5.0000E+00  
0.1323 7.7444E-05 8.7935E-01 1.0452E+00  
0.1545 4.3548E-04 7.7209E-01 4.5432E-01  
0.1768 1.2130E-03 6.7193E-01 2.6896E-01  
0.1990 2.5349E-03 5.7904E-01 1.8439E-01  
0.2213 4.5260E-03 4.9407E-01 1.3771E-01 

What I would like to do is re-sample the column 1 (index) values from a list, for example represented by:

indexList = numpy.linspace(0.11, 0.25, 8)

Then I need the values for columns 2, 3 and 4 to be linearly interpolated from the input DataFrame (it is always only my column 1 that I re-sample/reindex) - and if necessary extrapolated, as the min/max values for my list is not necessarily within my existing column 1 (index). However the key point is the interpolation part. I am quite new to python, but I was thinking using an approach like this:

1. output_df = DataFrame.reindex(index=indexList) - this will give me mainly NaN's for columns 2-4.
2. for index, row in output_df.iterrows()
   "function that calculates interpolated/extrapolated values from DataFrame and inserts them at correct row/column"

Somehow it feels like I should be able to use the .interpolate functionality, but I cannot figure out how. I cannot use it straightforward - it will be too inaccurate since most of my entries after re-indexing as mentioned in columns 2-4 will be NaN's; the interpolation should be done within the two closest values of my initial DataFrame. Any good tips anyone? (and if my format/intension is unclear, please let me know...)

Answer 1

Assuming column 1 is in the index, you can reindex your dataframe with the original values along with the list you created and then use interpolate to fill in the nan's.

df1 = df.reindex(df.index.union(np.linspace(.11,.25,8)))
df1.interpolate('index')

               2         3         4
0.1100  0.000000  1.000000  5.000000
0.1300  0.000069  0.891794  1.453094
0.1323  0.000077  0.879350  1.045200
0.1500  0.000363  0.793832  0.574093
0.1545  0.000435  0.772090  0.454320
0.1700  0.000976  0.702472  0.325482
0.1768  0.001213  0.671930  0.268960
0.1900  0.001999  0.616698  0.218675
0.1990  0.002535  0.579040  0.184390
0.2100  0.003517  0.537127  0.161364
0.2213  0.004526  0.494070  0.137710
0.2300  0.004526  0.494070  0.137710
0.2500  0.004526  0.494070  0.137710

Answer 2

Before we begin some spells:

import pandas as pd
import numpy

LENGTH=8

Let's start by loading your data (we'll change to csv cause it's easier):

x="""   1       2          3          4
0.1100 0.0000E+00 1.0000E+00 5.0000E+00
0.1323 7.7444E-05 8.7935E-01 1.0452E+00
0.1545 4.3548E-04 7.7209E-01 4.5432E-01
0.1768 1.2130E-03 6.7193E-01 2.6896E-01
0.1990 2.5349E-03 5.7904E-01 1.8439E-01
0.2213 4.5260E-03 4.9407E-01 1.3771E-01
"""
nx = ""
for l in x.split('\n'):
    nx += ','.join(l.split()) + '\n'
df= pd.read_csv(pd.compat.StringIO(nx))

Now, you want a new data frame interpolated on the same data but with an array of 8 values between 0.11 and 0.25:

indexList = numpy.linspace(0.11, 0.25, LENGTH)

We will use column one as the index, and reindex:

df_interpolated = df.reindex(df.index.union(indexList)).interpolate('index')
df_interpolated.head(LENGTH)

             1         2         3         4
0.00  0.110000  0.000000  1.000000  5.000000
0.11  0.112453  0.000009  0.986729  4.564972
0.13  0.112899  0.000010  0.984316  4.485876
0.15  0.113345  0.000012  0.981903  4.406780
0.17  0.113791  0.000013  0.979490  4.327684
0.19  0.114237  0.000015  0.977077  4.248588
0.21  0.114683  0.000016  0.974664  4.169492
0.23  0.115129  0.000018  0.972251  4.090396
0.25  0.115575  0.000019  0.969838  4.011300