Replace row values by condition if they are in certain time range

Question

I'm trying to replace certain values in a DataFrame row by two conditions. First they must be in a certain time range. Additionally, the value in this time range has to be in a list of values to be replaced.

My best attempt:

df = df[df.between_time('06:00', '20:00')].replace([0, 1, 2, 3], np.nan, inplace=True)

This is the error I get:

ValueError: Boolean array expected for the condition, not object

The DataFrame looks like this:

datetime	vehicles
2021-01-01 00:00:00	13.0
2021-01-01 00:15:00	9.0

And so on...

The main goal is to replace all values between 06:00 and 20:00 (8pm) with NaN, if they're <= 3.

Answer 1

import pandas as pd

Firstly convert your 'datetime' column into datetime dtype by(If it is already as datetime[ns] then ignore this step):

df['datetime']=pd.to_datetime(df['datetime'])

Then make your 'datetime' column as an index(If it is already as index then ignore this step):

df=df.set_index('datetime')

Now make use of between_time() method and apply() method:

resultdf=df.between_time('00:06:00', '00:20:00')['vehicles'].apply(lambda x:np.nan if x<=3 else x)

Finally:

resultdf.values.shape=(2,1)
df.loc[resultdf.index]=resultdf

Now if you print df you will get your desired output