删除python数据框中包含空格的行

Question

I imported a csv file to Python (Using Python data frame) and there are some missing values in a CSV file. In the data frame I have rows like following

> 08,63.40,86.21,63.12,72.78,,

I have tried everything to remove the rows containing the elements similar to the last element in the above data. Nothing works. I do not know if above is categorized as white space or empty string or what.

Here is what I have:

result = pandas.read_csv(file,sep='delimiter')
result[result!=',,']

This did not work. Then I have done following:

result.replace('  ', np.nan, inplace=True)
result.dropna(inplace=True)

This also did not work.

result = result.replace(r'\s+', np.nan, regex=True)

This also did not work. I still see the row containing the ,, element.

Also my dataframe is 100 by 1. When I import it from CSV file all the columns become 1.( I do not know if this helps)

Can anyone tell me how to remove rows containing ,, elements?

Answer 1

Also my dataframe is 100 by 1. When I import it from CSV file all the columns become 1

This is probably the key and IMHO is weird. When you import a csv in a pandas DataFrame you normally want each field to go in its own column, precisely to later be able to process that column values individually. So (still IMHO) the correct solution if to fix that.

Now to directly answer your (probably XY question), you do not want to remove rows containing blank or empty columns, because your row only contains one single column, but rows containing consecutive commas( ,, ). So you should use:

df.drop(df.iloc[0].str.contains(',,').index)

Answer 2

I think your code should work with a minor change:

result.replace('', np.nan, inplace=True)
result.dropna(inplace=True)

Answer 3

In case you have several rows in your CSV file, you can avoid the extra conversion step to NaN:

result = pandas.read_csv(file)
result = result[result.notnull().all(axis = 1)]

This will remove any row where there is an empty element.

However, your added comment explains that there is just one row in the CSV file, and it seems that the CSV reader shows some special behavior. Since you need to select the columns without NaN, I suggest these lines:

result = pandas.read_csv(file, header = None)
selected_columns = result.columns[result.notnull().any()]
result = result[selected_columns]

Note the option header = None with read_csv .