[英]Prevent Pandas read_csv from interpreting NA as NaN but retaining NaN for empty values
My question is related to this one .我的问题与此有关。 I have a file named 'test.csv' with 'NA' as a value for
region
.我有一个名为 'test.csv' 的文件,其中 'NA' 作为
region
的值。 I want to read this in as 'NA', not 'NaN'.我想把它读成“NA”,而不是“NaN”。 However, there are missing values in other columns in test.csv, which I want to retain as 'NaN'.
但是,test.csv 的其他列中存在缺失值,我想将其保留为“NaN”。 How can I do this?
我怎样才能做到这一点?
# test.csv looks like this:
Here's what I've tried:这是我尝试过的:
import pandas as pd
# This reads NA as NaN
df = pd.read_csv(test.csv)
df
region date expenses
0 NaN 1/1/2019 53
1 EU 1/2/2019 NaN
# This reads NA as NA, but doesn't read missing expense as NaN
df = pd.read_csv('test.csv', keep_default_na=False, na_values='_')
df
region date expenses
0 NA 1/1/2019 53
1 EU 1/2/2019
# What I want:
region date expenses
0 NA 1/1/2019 53
1 EU 1/2/2019 NaN
The problem with adding the argument keep_default_na=False
is that the second value for expenses
does not get read in as NaN
.添加参数
keep_default_na=False
的问题是expenses
的第二个值不会被读取为NaN
。 So if I then try pd.isnull(df['value'][1])
this is returned as False
.因此,如果我然后尝试
pd.isnull(df['value'][1])
这将返回为False
。
For me, this works:对我来说,这有效:
df = pd.read_csv('file.csv', keep_default_na=False, na_values=[''])
which gives:这使:
region date expenses
0 NA 1/1/2019 53.0
1 EU 1/2/2019 NaN
But I'd rather play safe, due to possible other NaN
in other columns, and do但我宁愿安全起见,因为其他列中可能存在其他
NaN
,并且做
df = pd.read_csv('file.csv')
df['region'] = df['region'].fillna('NA')
when specifying keep_default=False
all defaults values are not considered as nan so you should specify them:当指定
keep_default=False
时,所有默认值都不会被视为 nan,因此您应该指定它们:
use keep_default_na=False, na_values= ['', '#N/A', '#N/AN/A', '#NA', '-1.#IND', '-1.#QNAN', '-NaN', '-nan', '1.#IND', '1.#QNAN', 'N/A', 'NULL', 'NaN', 'n/a', 'nan', 'null']
使用
keep_default_na=False, na_values= ['', '#N/A', '#N/AN/A', '#NA', '-1.#IND', '-1.#QNAN', '-NaN', '-nan', '1.#IND', '1.#QNAN', 'N/A', 'NULL', 'NaN', 'n/a', 'nan', 'null']
This approach work for me:这种方法对我有用:
import pandas as pd
df = pd.read_csv('Test.csv')
co1 col2 col3 col4
a b c d e
NaN NaN NaN NaN NaN
2 3 4 5 NaN
I copied the value and created a list which are by default interpreted as NaN then comment out NA which I wanted to be interpreted as not NaN.我复制了该值并创建了一个默认解释为 NaN 的列表,然后注释掉我想要解释为非 NaN 的 NA。 This approach still treat other values as NaN except for NA.
此方法仍将除 NA 之外的其他值视为 NaN。
#You can also create your own list of value that should be treated as NaN and
# then pass the values to na_values and set keep_default_na=False.
na_values = ["",
"#N/A",
"#N/A N/A",
"#NA",
"-1.#IND",
"-1.#QNAN",
"-NaN",
"-nan",
"1.#IND",
"1.#QNAN",
"<NA>",
"N/A",
# "NA",
"NULL",
"NaN",
"n/a",
"nan",
"null"]
df1 = pd.read_csv('Test.csv',na_values=na_values,keep_default_na=False )
co1 col2 col3 col4
a b c d e
NaN NA NaN NA NaN
2 3 4 5 NaN
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.