如何在熊猫的数据框中删除 * 值？

Question

I am working with a data frame in pandas and some of the values in a certain column have * values. When I try to run a visual on that column using Seaborn I get the following error:

ValueError: could not convert string to float: '*'

I know what columns have * values in:

0      347
1      332
2      324
3      310
4      347
      ... 
163      *
164      *
165      *
166    310
167    319
Name: MeanScore, Length: 168, dtype: object

Answer 1

You can do to following. In my example, I will have the data frame df with two columns y and z which might potentially have a * in them.

import pandas as pd

df = pd.DataFrame({
    'x': [1, 2, 3, 4],
    'y': [1, 4, '*', 16],
    'z': [2, 3, 5, '*'],
})

df[(df['y'] != '*') & (df['z'] != '*')].head()

Answer 2

You can use replace function. As arguments just pass a dictionary according this scheme {I DON'T NEED IT: THIS IS EXACTLY WHAT I WANT}.

Ex:

df = pd.DataFrame({"column1": ["a", "b", "a"]})
print(df)

  column1
0       a
1       b
2       a

df["column1"].replace({"a": "x", "b": "y"}, inplace=True)
print(df)

  column1
0       x
1       y
2       x

Answer 3

Adding to_numpy gives a better performance.

# Example to reproduce solution
# We know that col_a and col_b contains '*'
df = pd.DataFrame(
    {
        "col_a": [1, 5, "*", 6, 8, "*"],
        "col_b": ["*", 6, 8, "*", 2, 4],
        "col_c": [1, 6, 8, 10, 2, 4],
    }
)
df = df[(df["col_a"].to_numpy() != "*") & (df["col_b"].to_numpy() != "*")]

And if you don't mind using numpy and the number of column is large you can use:

import numpy as np


def clean_asterisk(df, cols):
    """
    Drop asterisk in know columns
    
    Parameters:
    -----------
    df : pd.DataFrame
    DataFrame we want to clean
    
    cols : str or list of strings
    List of known columns with asterisks
    
    Returns:
    --------
    df : pd.DataFrame
    DataFrame cleaned without asterisk
    """
    if len(cols) == 0:
        raise ValueError(
            "Pass at least a list of one element or a string with one character"
        )
    if len(cols) == 1 or isinstance(cols, str):
        try:
            df = df[df[cols].to_numpy() != "*"]
            return df
        except KeyError:
            print(f"Column {cols} must be in the DataFrame")
    try:
        df = df[np.bitwise_and.reduce([df[col] != "*" for col in cols])]
        return df
    except KeyError:
        print(f"Column {cols} must be in the DataFrame")

df = clean_asterisk(df,["col_a","col_b"])

The last approach is much more scalable but too complex for small examples.