[英]pandas read_csv does not capture final (unnamed) column into dataframe
I am trying to read a csv file in the following format 我正在尝试读取以下格式的csv文件
myHeader
myJunk
myDate
A, B, C, D
, b, c, d
dataA, dataB, dataC, dataD, EXTRA_INFO_STRING
dataA, dataB, dataC, dataD, EXTRA_INFO_STRING
dataA, dataB, dataC, dataD, EXTRA_INFO_STRING
When I create my data frame using 当我使用创建数据框时
dlogframe = pd.read_csv(myPath, header=3)
I get the following error (my data is more complex than above example, but functionally identical) 我收到以下错误(我的数据比上面的示例复杂,但功能相同)
pandas._parser.CParserError: Error tokenizing data. C error: Expected 393 fields in line 9, saw 394
How can I give the EXTRA_INFO column a name and have those strings included in my dataframe? 如何给EXTRA_INFO列命名,并在数据框中包含这些字符串?
[EDIT] [编辑]
I figured out how to skip the troublesome row, but now the data is not aligned properly 我想出了如何跳过麻烦的行,但是现在数据没有正确对齐
from StringIO import StringIO
s = """myHeader
myJunk
myDate
A, B, C, D
, b, c, d
dataA, dataB, dataC, dataD, EXTRA_INFO_STRING
dataA, dataB, dataC, dataD, EXTRA_INFO_STRING
dataA, dataB, dataC, dataD, EXTRA_INFO_STRING"""
df = pd.read_csv(StringIO(s), header=3, skiprows=[4])
>>print df
A B C D
dataA dataB dataC dataD EXTRA_INFO_STRING
dataA dataB dataC dataD EXTRA_INFO_STRING
dataA dataB dataC dataD EXTRA_INFO_STRING
What I want is: 我想要的是:
A B C D MY_INFO
dataA dataB dataC dataD EXTRA_INFO_STRING
dataA dataB dataC dataD EXTRA_INFO_STRING
dataA dataB dataC dataD EXTRA_INFO_STRING
If only row after header is missing EXTRA_INFO_STRING
, you can load column names and data separately: 如果标头后仅缺少
EXTRA_INFO_STRING
行,则可以分别加载列名和数据:
from StringIO import StringIO
df = pd.read_csv(StringIO(s), header=None, skiprows=5)
Following code (may be not a very elegant one) will load column names: 以下代码(可能不是很优雅)将加载列名:
df.columns = pd.read_csv(StringIO(s), header=None,
nrows=1, skiprows=3).T.append(['MY_INFO'])[0]
# 0 A B C D MY_INFO
# 0 dataA dataB dataC dataD EXTRA_INFO_STRING
# 1 dataA dataB dataC dataD EXTRA_INFO_STRING
# 2 dataA dataB dataC dataD EXTRA_INFO_STRING
Data used in test: 测试中使用的数据:
s = """myHeader
myJunk
myDate
A, B, C, D
, b, c, d
dataA, dataB, dataC, dataD, EXTRA_INFO_STRING
dataA, dataB, dataC, dataD, EXTRA_INFO_STRING
dataA, dataB, dataC, dataD, EXTRA_INFO_STRING"""
How about: 怎么样:
df = pd.read_csv(StringIO(s), skiprows=5, header = None, index_col = False)
df.columns = list("ABCDE")
Sometimes if you have problem with read_csv numeric conversions you could add dtype=object into read_csv call and deal with conversions later on your own using DataFrame.astype. 有时,如果您对read_csv数字转换有疑问,可以将dtype = object添加到read_csv调用中,然后稍后使用DataFrame.astype自行处理转换。
Here is something I tried that seems to get data in the format that you want. 这是我尝试过的方法,似乎可以获取所需格式的数据。 Basic idea is 'ignore all problematic rows' (that's possible if you know about the file structure).
基本思想是“忽略所有有问题的行”(如果您知道文件结构,则有可能)。
x = pd.read_csv(StringIO.StringIO(s), names=['a', 'b', 'c', 'd', 'more_info'], header=None, skiprows=5)
This gives output in the format you desire. 这将以您想要的格式输出。
My experience has been with read_csv
, one has to try a few combinations before one gets what one wants. 我的经验是使用
read_csv
,人们必须尝试几种组合才能获得所需的东西。
Hope this helps. 希望这可以帮助。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.