Create a vector with values from a Numpy array selected according to criteria in a Pandas DataFrame

Question

I am working with a pandas df that contains two columns with integers. For each data of the df, I would like to select these two integers, use them as [row,column] pairs to extract values from a np.array and create a new np.array with the extracted values.

In more detail, my df contains the following entries:

             State  FutureState
DATE                                                         
1947-10-01       0            0
1948-01-01       0            1
1948-04-01       1            1
1948-07-01       1            1

For each Date , I would like to select the [State,FutureState] pair and extract the corresponding [row,column] item from the following np.array, called P :

array([[ 0.7,  0.3],
       [ 0.4,  0.6]])

With these values, I would like to create a new np.array called Transition , which contains of the following values:

[P[0,0],P[0,1],P[1,1],P[1,1]] = [0.7, 0.3, 0.6, 0.6] 

The pairs [0,0], [0,1], [1,1] [1,1] used as index for the array P are the values for [State,FutureState] for each date ( 1947-10-01, 1948-01-01 , 1948-04-01, 1948-07-01 ).

I already tried to solve my problem in a lot of different ways but to no avail. Can somebody kindly suggest how to successfully create the Transition vector?

Answer 1

How about this?

df.apply(lambda x:P[x[0],x[1]], axis=1)

It does what you describe, go row-wise (so apply over axis=1 ) along df and use the entries as index for selecting in P .

Answer 2

try this:

p[df.State, df.FutureState]

Here is the full code:

import io
import pandas as pd
import numpy as np

txt = """             State  FutureState                                                        
1947-10-01       0            0
1948-01-01       0            1
1948-04-01       1            1
1948-07-01       1            1"""

df = pd.read_csv(io.BytesIO(txt), delim_whitespace=True)
p = np.array([[ 0.7,  0.3], [ 0.4,  0.6]])
p[df.State, df.FutureState]