将 pandas dataframe 拆分为多个 csv 文件，将组保持在一起

Question

I have a 1M lines dataframe which consists of the following: I do not really know the list of keys specifically so I cannot say, filter after Key ABCDE.

Key;Value

A;1
A;2
B;3
B;4
B;5
C;6
C;7
D;8

I would like to split this one into 3 dataframes keeping the groups together So the result should be

A;1
A;2

B;3
B;4
B;5
C;6
C;7

D;8

So I would like to split it but it needs to keep the groups together, you cannot do this for example

A;1
A;2
B;3

B;4
B;5
C;6

C;7
D;8

So I would like to split it after a group is finished.

I tried a bit with the pandas groupby function but no really sure how to do the split when you have all the groups together.

The split can be quite random eg. every 1K lines It does not really matter its more that the groups need to be together.

Answer 1

There are a few ways, using groupby for example. Here's one way.

import pandas as pd

df = pd.DataFrame({"key":["A","A","B","B","C","C","D","D"],
                   "value":[1,2,3,4,5,6,7,8]})

df.loc[df["key"] == "A", :].to_csv("filename_A.csv")
df.loc[(df["key"] == "B") | (df["key"] == "C"), :].to_csv("filename_BC.csv")
df.loc[df["key"] == "D", :].to_csv("filename_D.csv")

Answer 2

You can use numpy.array_split for split by groups and them select values in Series.isin and boolean indexing :

N = 3
for i, key in enumerate(np.array_split(df['Key'].unique(), N)):
    print (key)
    df[df['Key'].isin(key)].to_csv(f'file{i}.csv', index=False)