[英]How to skip blank lines with read_fwf in pandas?
I use pandas.read_fwf()
function in Python pandas 0.19.2 to read a file fwf.txt
that has the following content: 我在Python pandas 0.19.2中使用
pandas.read_fwf()
函数来读取具有以下内容的文件fwf.txt
:
# Column1 Column2
123 abc
456 def
#
#
My code is the following: 我的代码如下:
import pandas as pd
file_path = "fwf.txt"
widths = [len("# Column1"), len(" Column2")]
names = ["Column1", "Column2"]
data = pd.read_fwf(filepath_or_buffer=file_path, widths=widths,
names=names, skip_blank_lines=True, comment="#")
The printed dataframe is like this: 打印的数据框如下:
Column1 Column2
0 123.0 abc
1 NaN NaN
2 456.0 def
3 NaN NaN
It looks like the skip_blank_lines=True
argument is ignored, as the dataframe contains NaN's. 看起来像
skip_blank_lines=True
参数被忽略,因为数据帧包含NaN。
What should be the valid combination of pandas.read_fwf()
arguments that would ensure the skipping of blank lines? 什么应该是
pandas.read_fwf()
参数的有效组合,以确保跳过空行?
import io
import pandas as pd
file_path = "fwf.txt"
widths = [len("# Column1 "), len("Column2")]
names = ["Column1", "Column2"]
class FileLike(io.TextIOBase):
def __init__(self, iterable):
self.iterable = iterable
def readline(self):
return next(self.iterable)
with open(file_path, 'r') as f:
lines = (line for line in f if line.strip())
data = pd.read_fwf(FileLike(lines), widths=widths, names=names,
comment='#')
print(data)
prints 版画
Column1 Column2
0 123 abc
1 456 def
with open(file_path, 'r') as f:
lines = (line for line in f if line.strip())
defines a generator expression (ie an iterable) which yields lines from the file with blank lines removed. 定义一个生成器表达式(即一个可迭代的),它从文件中产生一行,并删除空白行。
The pd.read_fwf
function can accept TextIOBase
objects. pd.read_fwf
函数可以接受TextIOBase
对象。 You can subclass TextIOBase
so that its readline
method returns lines from an iterable: 您可以
TextIOBase
子类,以便其readline
方法从iterable返回行:
class FileLike(io.TextIOBase):
def __init__(self, iterable):
self.iterable = iterable
def readline(self):
return next(self.iterable)
Putting these two together gives you a way to manipulate/modify lines of a file before passing them to pd.read_fwf
. 将这两者放在一起为您提供了一种在将文件传递给
pd.read_fwf
之前操作/修改文件行的pd.read_fwf
。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.