简体   繁体   中英

How to filter dataframe based on values in pyspark/python?

I have a dataframe like below. I want to read the dataframe and filter the records based on start time and store in different dataframes.

INPUT DF

name      start_time
AA        2022-11-16
AAA       2022-11-15
BBB       2022-11-14

For eg: I need to store each record based on start time, which means all, 16 th date start time records should go to one dataframe and so on.

OUTPUT DF

df1 = ["Store 2022-11-16 record"]
df2 = ["Store 2022-11-15 record"]
df3 = ["Store 2022-11-14 record"]

Well, technially a duplicate but idk how to report that but I think this works:

df = pd.DataFrame({"name" : ["AA", "AAA", "BBB"], 
"start_time" : ["2022-11-16"," 2022-11-15", "2022-11-14"]})

dfs = dict(tuple(df.groupby('start_time')))

dfs

you can select each DataFrame by the start time:

print (dfs['2022-11-14''])

    name    start_time
2   BBB 2022-11-14

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM