在日期时间索引的熊猫数据框中选择固定时间间隔的行

Question

I have a pandas dataframe indexed by DateTime from hour "00:00:00" until hour "23:59:00" (increments by minute, seconds not counted).

in: df.index
out: DatetimeIndex(['2018-10-08 00:00:00', '2018-10-08 00:00:00',
           '2018-10-08 00:00:00', '2018-10-08 00:00:00',
           '2018-10-08 00:00:00', '2018-10-08 00:00:00',
           '2018-10-08 00:00:00', '2018-10-08 00:00:00',
           '2018-10-08 00:00:00', '2018-10-08 00:00:00',
           ...
           '2018-10-08 23:59:00', '2018-10-08 23:59:00',
           '2018-10-08 23:59:00', '2018-10-08 23:59:00',
           '2018-10-08 23:59:00', '2018-10-08 23:59:00',
           '2018-10-08 05:16:00', '2018-10-08 07:08:00',
           '2018-10-08 13:58:00', '2018-10-08 09:30:00'],
          dtype='datetime64[ns]', name='DateTime', length=91846, freq=None)

Now I want to choose specific intervals, say every 1 minute, or every 1 hour, starting from "00:00:00" and retrieve all the rows that interval apart consecutively.

I can grab entire intervals, say the first hour interval, with

df.between_time("01:00:00","00:00:00")

But I want to be able to

(a) get only all the times that are a specific intervals apart (b) get all the 1-hour intervals without having to manually ask for them 24 times. How do I increment the DatetimeIndex inside the between_time command? Is there a better way than that?

Answer 1

I would solve this problem with masking rather than making new dataframes. For example you can add a column df['which_one'] and set different numbers for each subset. Then you can access the subset by calling df[df['which_one']==x] where x is the subset you want to select. You can still do other conditional statements and just about everything else that Pandas had to offer by access the data this way.

PS There are other methods to access data that might be faster. I just used what I'm most comfortable with another way would be df[df['which_one'].eq(x)] .

Answer 2

If you are deadset on dataframes I would suggest doing so with a dictionary of dataframes such as:

import pandas as pd

dfdict={}

for i in range(0,10):
    dfdict[i]=pd.DataFrame()

print(dfdict)

as you will see they are indeed dfs

out[1]
{0: Empty DataFrame
Columns: []
Index: [], 1: Empty DataFrame
Columns: []
Index: [], 2: Empty DataFrame
Columns: []
Index: [], 3: Empty DataFrame
Columns: []
Index: [], 4: Empty DataFrame
Columns: []
Index: [], 5: Empty DataFrame
Columns: []
Index: [], 6: Empty DataFrame
Columns: []
Index: [], 7: Empty DataFrame
Columns: []
Index: [], 8: Empty DataFrame
Columns: []
Index: [], 9: Empty DataFrame
Columns: []
Index: []}

Although as others have suggested there might be a more practical approach to solve your problem (difficult to say without more specifics of the issue)