在 Python 中，我如何构建一个循环，允许我读取 txt 文件（制表符分隔）并将每 1000 行存储为自己的数据帧？

Question

Below is a sample of my data, there is ten row header, then exactly 1000 rows of data, then it repeats for 30 cycles (these are trials for a lab experiment). I have 8 of these files with the same format and I would like to extract each batch so I can then do some stuff. How do I make a loop that creates a new dataframe each time to store the new rows?

Channels    1   
Samples 1000    
Date    2020/02/12  
Time    10:11:36.6395705499426038443    
Y_Unit_Label    Volts   
X_Dimension Time    
X0  0.0000000000000000E+0   
Delta_X 0.001000    
***End_of_Header***     
X_Value Voltage Comment
0.000000    4.930675    4.96V\0A69.0 cm\0A6.9 degrees
0.001000    4.934949
0.002000    4.931990
0.003000    4.923443

I'm trying to do something like the code below, but I can't figure out to get pandas to create a new dataframe for each iteration.

collection=['Rawdata01.txt','Rawdata02.txt','Rawdata03.txt']
result = pd.DataFrame()
for i in collection:
    j=0
    mydf = pd.read_csv(i,sep='\t',header=(0),index_col=False)
    for row in mydf.iterrows():
        result = csv[1000*j + 10*(j+1):1000*(j+1) + 10*(j+1)] # how to get it to make newdataframes
        print(result.head())
        j=j+1

I've gotten pretty close, but I'm stuck on how to proceed to getting either separate dataframes for each batch or one big one. At this point, either will work. Any help on this matter would be greatly appreciated.

Answer 1

After this line:

mydf = pd.read_csv(i,sep='\t',header=(0),index_col=False)

you already have the tab-delimited data in single dataframe. To break it into 1000-rows each, you can try this:

sub_frames = [mydf.iloc[startrow:startrow+1000] for startrow in range(0, len(mydf), 1000)]

then you have a list of dataframes, each has 1000 rows (except possibly the last one). The iloc of a dataframe is to extract rows from a bigger dataframe.