[英]Remove rows with empty lists from pandas data frame
I have a data frame with some columns with empty lists and others with lists of strings:我有一个数据框,其中一些列带有空列表,而其他列带有字符串列表:
donation_orgs donation_context
0 [] []
1 [the research of Dr. ...] [In lieu of flowers , memorial donations ...]
I'm trying to return a data set without any of the rows where there are empty lists.我正在尝试返回一个没有任何空列表行的数据集。
I've tried just checking for null values:我试过只检查空值:
dfnotnull = df[df.donation_orgs != []]
dfnotnull
and和
dfnotnull = df[df.notnull().any(axis=1)]
pd.options.display.max_rows=500
dfnotnull
And I've tried looping through and checking for values that exist, but I think the lists aren't returning Null or None like I thought they would:而且我尝试循环并检查存在的值,但我认为列表没有像我想象的那样返回 Null 或 None :
dfnotnull = pd.DataFrame(columns=('donation_orgs', 'donation_context'))
for i in range(0,len(df)):
if df['donation_orgs'].iloc(i):
dfnotnull.loc[i] = df.iloc[i]
All three of the above methods simply return every row in the original data frame.=上述所有三种方法都只是简单地返回原始数据框中的每一行。=
To avoid converting to str
and actually use the list
s, you can do this:为避免转换为
str
并实际使用list
,您可以这样做:
df[df['donation_orgs'].map(lambda d: len(d)) > 0]
It maps the donation_orgs
column to the length of the lists of each row and keeps only the ones that have at least one element , filtering out empty lists.它将
donation_orgs
列映射到每行列表的长度,并只保留至少有一个元素的列,过滤掉空列表。
It returns它返回
Out[1]:
donation_context donation_orgs
1 [In lieu of flowers , memorial donations] [the research of Dr.]
as expected.正如预期的那样。
You could try slicing as though the data frame were strings instead of lists:您可以尝试切片,就好像数据框是字符串而不是列表一样:
import pandas as pd
df = pd.DataFrame({
'donation_orgs' : [[], ['the research of Dr.']],
'donation_context': [[], ['In lieu of flowers , memorial donations']]})
df[df.astype(str)['donation_orgs'] != '[]']
Out[9]:
donation_context donation_orgs
1 [In lieu of flowers , memorial donations] [the research of Dr.]
您可以使用以下单线:
df[(df['donation_orgs'].str.len() != 0) | (df['donation_context'].str.len() != 0)]
Assuming that you read data from a CSV, the other possible solution could be this:假设您从 CSV 读取数据,另一个可能的解决方案可能是:
import pandas as pd
df = pd.read_csv('data.csv', na_filter=True, na_values='[]')
df.dropna()
na_filter
defines additional string to recognize as NaN. na_filter
定义附加字符串以识别为 NaN。 I tested this on pandas-0.24.2
.我在
pandas-0.24.2
上对此进行了测试。
可能是数据类型不同,这可能会有所帮助
df[df.astype(str)['donation_orgs'] != '[]']
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.