无法从 dtype('

Question

I am using numpy interp to interpolate datapoint but was given Cannot cast array data from dtype('

Code snippet:

import pandas as pd
import numpy as np
def interpolate_fwd_price(row, fx):
    res = np.interp(row['SA_M'], fx['TENOR_DT'], fx['RATE'])
    return res

df = pd.DataFrame({'SA_M': ['2018-02-28','2018-03-10']})
df['SA_M'] = pd.to_datetime(df['SA_M'])
data = pd.DataFrame({'TENOR_DT': ['2017-02-09','2017-03-02','2017-04-03','2017-05-02'], 'RATE':[1.0, 1.2, 1.5, 1.8]})
data['TENOR_DT'] = pd.to_datetime(data['TENOR_DT'])
df['PRICE'] = df.apply(interpolate_fwd_price, fx=data, axis=1)

I did some search and could not figure out what is causing the error. Appreciate your input.

Make some change and it works for interpolating the datetime difference instead of datetime directly. Would still be interested to know why it did not work for interpolating datetime directly.

def interpolate_fwd_price(row, fx):
    fx['DT'] = (fx['TENOR_DT'] - row(['SA_M'])).dt.days
    res = np.interp(0, fx['DT'], fx['RATE'])
    return res

Answer 1

In [92]: data = pd.DataFrame({'TENOR_DT': ['2017-02-09','2017-03-02','2017-04-03','2017-05-02'], 'RATE':[1.0, 1.2, 1.5, 1.8]})
In [93]: data        # object dtype with strings
Out[93]: 
   RATE    TENOR_DT
0   1.0  2017-02-09
1   1.2  2017-03-02
2   1.5  2017-04-03
3   1.8  2017-05-02
In [94]: data['TENOR_DT'] = pd.to_datetime(data['TENOR_DT'])
In [95]: data
Out[95]: 
   RATE   TENOR_DT
0   1.0 2017-02-09
1   1.2 2017-03-02
2   1.5 2017-04-03
3   1.8 2017-05-02
In [96]: data['TENOR_DT']
Out[96]: 
0   2017-02-09
1   2017-03-02
2   2017-04-03
3   2017-05-02
Name: TENOR_DT, dtype: datetime64[ns]

The array version of the dates:

In [98]: dt = data['TENOR_DT'].values
In [99]: dt
Out[99]: 
array(['2017-02-09T00:00:00.000000000', '2017-03-02T00:00:00.000000000',
       '2017-04-03T00:00:00.000000000', '2017-05-02T00:00:00.000000000'],
      dtype='datetime64[ns]')

It can be cast to float using the default unsafe :

In [100]: dt.astype(float)
Out[100]: array([1.4865984e+18, 1.4884128e+18, 1.4911776e+18, 1.4936832e+18])
In [101]: dt.astype(float, casting='safe')
TypeError: Cannot cast array from dtype('<M8[ns]') to dtype('float64') according to the rule 'safe'

My guess is that np.interp is using the safe casting to convert those datetime values to floats.

I haven't tried to do interp with dates before, so can only suggest some fixes. First your dates only differ by day, so we don't need the full ns resolution:

In [107]: dt.astype('datetime64[D]')
Out[107]: 
array(['2017-02-09', '2017-03-02', '2017-04-03', '2017-05-02'],
      dtype='datetime64[D]')

It still won't allow safe casting, but the 'unsafe' casting produces reasonable looking numbers. You might be able to use those in the interpolation.

In [108]: dt.astype('datetime64[D]').astype(int)
Out[108]: array([17206, 17227, 17259, 17288])