[英]How to split a CSV file on blank rows
我有一个CSV文件,它在两个空白行后启动一个新主题。 我想将此文件拆分为两个不同的文件。 我怎样才能做到这一点?
................
................
Biology I
BGS Shivamogga I PUC Exam Results
Student Exam # Questions Correct Answers Score %
ADARSHGOUDA M MUDIGOUDAR Biology I - Chapter 1 35 23 65.70%
ADARSHGOUDA M MUDIGOUDAR Biology I - Chapter 1 35 29 82.90%
ADARSHGOUDA M MUDIGOUDAR Biology I - Chapter 1 35 32 91.40%
.
.
.
.
................
................
Chemistry I
BGS Shivamogga I PUC Exam Results
Student Exam # Questions Correct Answers Score %
AISHWARYA P Chemistry I - Chapter 1 29 20 69.00%
MAHARUDRASWAMY M S Chemistry I - Chapter 1 29 14 48.30%
NIKHIL B Chemistry I - Chapter 1 29 20 69.00%
我曾尝试使用dropnas
和skiprows
来分割数据帧,但我不想硬编码行数。 我想基于前两个空白行进行拆分。
我会按照以下方式做点什么:
with open('input.txt','r') as input_file:
data_str = input_file.read()
data_array = data_str.split('\n\n') # Split on all instances of double new lines
for i, smaller_data in enumerate(data_array):
with open(f'new_file_{i}.txt','w') as new_data_file:
new_data_file.write(smaller_data)
我只是使用csv
模块,将csv.reader()
中的行应对到csv.writer()
对象,并csv.writer()
保持连续空行的计数。 每次找到多个空白行时,将一个写入对象替换为一个新文件。
您可以使用any()
函数检测空行,因为空行只包含空字符串或根本没有值:
isblank = not any(row)
假设同一目录中的编号文件就足够了,这应该有效:
import csv
from pathlib import Path
def gen_outputfiles(outputdir, basefilename):
"""Generate open files ready for CSV writing, in outputdir using basefilename
Numbers are inserted between the basefilename stem and suffix; e.g.
foobar.csv becomes foobar001.csv, foobar002.csv, etc.
"""
outputbase = Path(basefilename)
outputstem, outputsuffix = outputbase.stem, outpubase.suffix
counter = 0
while True:
counter += 1
yield outputdir / f'{outputstem}{counter:03d}{outputsuffix}'.open(mode='w', newline='')
def split_csv_on_doubleblanks(inputfilename, basefilename=None, **kwargs):
"""Copy CSV rows from inputfilename to numbered files based on basefilename
A new numbered target file is created after 2 or more blank rows have been
read from the input CSV file.
"""
inputpath = Path(inputfilename)
outputfiles = gen_outputfiles(inputpath.parent, basefilename or inputpath.name)
with inputpath.open(newline='') as inputfile:
reader = csv.reader(inputfile, **kwargs)
outputfile = next(outputfiles())
writer = csv.writer(outputfile, **kwargs)
blanks = 0
try:
for row in reader:
isblank = not any(row)
if not isblank and blank > 1:
# skipped more than one blank row before finding a non-blank
# row. Open a new output file
outputfile.close()
outputfile = next(outputfile)
writer = csv.writer(outputfile, **kwargs)
blank = blank + 1 if isblank else 0
writer.writerow(row)
finally:
if not outputfile.closed:
outputfile.close()
请注意,我也会复制空白行,因此您的文件最终会有多个空白行。 这可以通过用空白行列表替换blanks
计数器来解决,以便在想要重置计数器时写出到编写器对象,并且该列表中只有一个元素。 这样就可以保留单个空白行。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.