Splitting a dataframe into many smaller data frames evenly-ish

Question

I have the following frame called df which is 65 items long.

   Name  Data
0  Name1 Data1
1  Name2 Data2
2  Name3 Data3
....

I want to split it into 30 data frames as evenly as possible.

So with a length of 65, I want there to be 5 frames of length 3 and 25 of length 2 (which adds up to 65)

I use the following function:

def chunk(seq, size):
    return (seq[pos:pos + size] for pos in range(0, len(seq), size))

n = 30 #number of files

length = len(df)

counter=0

for df_chunk in chunk(frame, int(length / n) + (length % n > 0)):
    counter+=1
    df_chunk.to_csv(f"path/to/file{counter}.csv")

But I only get 21 files which are, 3 in length and 1 file which is 2 in length instead of 5 files which are 3 in length and 25 which are 2 in length.

Anyone has any ideas on how I can achieve what I want?

Answer 1

Use, np.array_split , from the documentation it says:

For an array of length l that should be split into n sections, it returns l % n sub-arrays of size l//n + 1 and the rest of size l//n. :

for counter, df_chunk in enumerate(np.array_split(df, 30), 1):
    df_chunk.to_csv(f"path/to/file{counter}.csv")