根据值的类型过滤Pandas Dataframe中的数据

Question

I am hardly trying to filter my dataframe set using the.loc function, with a condition based on the type of the data in one of my column.

My goal is to apply (with.apply) a function on a column only on rows with a certain type.

I tried to use "dtype", but my column has values with 2 different types. So I'm only getting "object".

So, when I do: print(df.info(verbose=True)) I get this:

 #   Column               Non-Null Count  Dtype 
---  ------               --------------  ----- 
 0   address              26419 non-null  object
.
.
.

Here is what I am trying to run:

import ipaddress as ipa
.
.
.
    df.loc['EXCEPTION'] = df.loc[isinstance(df['address'], ipa.IPv4Network)].apply(
        return_row_with_exception,
        axis=1)

It's supposed to update only the column 'EXCEPTION' on the dataframe 'df', only on rows for which the data in the column 'address' is IPv4Network type. The function 'return_row_with_exception' returns the string contents of 'EXCEPTION' for each row, based on a rule using other columns of the row.

Unfortunately, I am getting this error, can someone help me on this:D

Traceback (most recent call last):
  File "pythonProject1111\venv\lib\site-packages\pandas\core\indexes\base.py", line 2895, in get_loc
    return self._engine.get_loc(casted_key)
  File "pandas\_libs\index.pyx", line 70, in pandas._libs.index.IndexEngine.get_loc
  File "pandas\_libs\index.pyx", line 98, in pandas._libs.index.IndexEngine.get_loc
  File "pandas\_libs\index_class_helper.pxi", line 93, in pandas._libs.index.Int64Engine._check_type
KeyError: False

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "pythonProject1111\main.py", line 14, in <module>
    abc = lib_read_from_imap.process_abc(abc)
  File "pythonProject1111\libs\read_from_abc.py", line 178, in process_abc
    df_file_abc = scaexc.fill_scan_exception(df_file_abc)
  File "pythonProject1111\libs\process_scan_exception.py", line 80, in fill_scan_exception
    print(df.loc[isinstance(df['address'], ipa.IPv4Network)])
  File "pythonProject1111\venv\lib\site-packages\pandas\core\indexing.py", line 879, in __getitem__
    return self._getitem_axis(maybe_callable, axis=axis)
  File "pythonProject1111\venv\lib\site-packages\pandas\core\indexing.py", line 1110, in _getitem_axis
    return self._get_label(key, axis=axis)
  File "pythonProject1111\venv\lib\site-packages\pandas\core\indexing.py", line 1059, in _get_label
    return self.obj.xs(label, axis=axis)
  File "pythonProject1111\venv\lib\site-packages\pandas\core\generic.py", line 3491, in xs
    loc = self.index.get_loc(key)
  File "pythonProject1111\venv\lib\site-packages\pandas\core\indexes\base.py", line 2897, in get_loc
    raise KeyError(key) from err
KeyError: False

Many thanks!!

Answer 1

As you mention, dtypes does work if you have multiple types. Here is what you could do instead:

employees = [('jack', 34, 'Sydney', 155),
            ('Riti', 31, 'Delhi', 177.5),
            ('Aadi', 16, 'Mumbai', 81),
            ('Mohit', 31, 45, 167),
            ('Veena', 12, 'Delhi', 'Serge'),
            ('Shaunak', 35, 'Mumbai', 135),
            ('Shaun', 35, 'Colombo', 111)
            ]
empDfObj = pd.DataFrame(empoyees, columns=['Name', 'Age', 'City', 'Marks'])
empDfObj.applymap(type).apply(pd.value_counts).fillna(0)

where you use .apply .

Giveing you

                 Name  Age  City  Marks
<class 'str'>     7.0  0.0   6.0      1
<class 'int'>     0.0  7.0   1.0      5
<class 'float'>   0.0  0.0   0.0      1

You even get a count on them:-)