[英]python pandas read_csv delimiter in column data
I'm having this type of CSV file: 我有这种类型的CSV文件:
12012;My Name is Mike. What is your's?;3;0
1522;In my opinion: It's cool; or at least not bad;4;0
21427;Hello. I like this feature!;5;1
I want to get this data into da pandas.DataFrame
. 我想将此数据放入da pandas.DataFrame
。 But read_csv(sep=";")
throws exceptions due to the semicolon in the user generated message column in line 2 (In my opinion: It's cool; or at least not bad). 但是由于第2行中用户生成的消息列中的分号,因此read_csv(sep=";")
会引发异常(在我看来:这很酷;或者至少还不错)。 All remaining columns constantly have numeric dtypes. 其余所有列始终具有数字dtype。
What is the most convenient method to manage this? 最方便的管理方法是什么?
Dealing with unquoted delimiters is always a nuisance. 处理不带引号的定界符总是很麻烦。 In this case, since it looks like the broken text is known to be surrounded by three correctly-encoded columns, we can recover. 在这种情况下,由于已知损坏的文本看起来被三个正确编码的列包围,因此我们可以进行恢复。 TBH, I'd just use the standard Python reader and build a DataFrame once from that: TBH,我只需要使用标准的Python阅读器并从中构建一个DataFrame:
import csv
import pandas as pd
with open("semi.dat", "r", newline="") as fp:
reader = csv.reader(fp, delimiter=";")
rows = [x[:1] + [';'.join(x[1:-2])] + x[-2:] for x in reader]
df = pd.DataFrame(rows)
which produces 产生
0 1 2 3
0 12012 My Name is Mike. What is your's? 3 0
1 1522 In my opinion: It's cool; or at least not bad 4 0
2 21427 Hello. I like this feature! 5 1
Then we can immediately save it and get something quoted correctly: 然后,我们可以立即保存它并得到正确引用的内容:
In [67]: df.to_csv("fixedsemi.dat", sep=";", header=None, index=False)
In [68]: more fixedsemi.dat
12012;My Name is Mike. What is your's?;3;0
1522;"In my opinion: It's cool; or at least not bad";4;0
21427;Hello. I like this feature!;5;1
In [69]: df2 = pd.read_csv("fixedsemi.dat", sep=";", header=None)
In [70]: df2
Out[70]:
0 1 2 3
0 12012 My Name is Mike. What is your's? 3 0
1 1522 In my opinion: It's cool; or at least not bad 4 0
2 21427 Hello. I like this feature! 5 1
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.