Inplace Modifying a Column in a DataFrame

Question

Edit: I've since realised that when accessing with .loc I get no error but a warning, so the code actually works.

I have a dataset that needs some cleaning. One of the things I needed to do was to remove all transactions of more than 50000kn. It happens that the cost bruto_prihod is in string format and that the transactions transakcija_sifra span multiple rows. An additional inconvenience is that the decimal formatting is with a comma (we use a decimal comma in Croatian) instead of a dot.

This is my code before the task:

    import pandas as pd
    import numpy as np

    diversus = pd.read_csv(r"C:\Users\dagejev\Desktop\excel\diversus4_saved.csv", encoding="cp1252", sep=";")

    df = diversus.copy()

    df_filter = df[(df['faza'] == 'A') &
                   ( (df['grupa_id'] == 'W ODIJELO') | (df['grupa_id'] == 'ODIJELO') ) &
                   ( (df['vp_kupac_id']!= 591) & (df['vp_kupac_id']!= 333) & (df['vp_kupac_id']!= 332) ) &
                   ( (df['brand_naziv'] == 'JOOP') | (df['brand_naziv'] == 'JOOP!') ) &
                   (df['vrsta_robe']=='ROBA') &
                   (df['divizija'] == 'MEN')]
    print('df: ', len(df), 'filtered: ', len(df_filter))

Here's the relevant part:

#This is what I tried first

df_filter['bruto_prihod'] = df_filter['bruto_prihod'].str.replace(',','.')

However, it doesn't work:

C:\Users\dagejev\AppData\Local\Continuum\anaconda3\lib\site-packages\ipykernel_launcher.py:1: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
  """Entry point for launching an IPython kernel.

And neither does the suggested alternative:

    df_filter[:,'bruto_prihod'] = df_filter['bruto_prihod'].str.replace(',','.')

    ---------------------------------------------------------------------------
    TypeError                                 Traceback (most recent call last)
    <ipython-input-82-0e27bdc0e78f> in <module>
    ----> 1 df_filter[:,'bruto_prihod'] = df_filter['bruto_prihod'].str.replace(',','.')

    ~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\frame.py in __setitem__(self, key, value)
       3368         else:
       3369             # set column
    -> 3370             self._set_item(key, value)
       3371 
       3372     def _setitem_slice(self, key, value):

    ~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\frame.py in _set_item(self, key, value)
       3443 
       3444         self._ensure_valid_index(value)
    -> 3445         value = self._sanitize_column(key, value)
       3446         NDFrame._set_item(self, key, value)
       3447 

    ~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\frame.py in _sanitize_column(self, key, value, broadcast)
       3659 
       3660         # broadcast across multiple columns if necessary
    -> 3661         if broadcast and key in self.columns and value.ndim == 1:
       3662             if (not self.columns.is_unique or
       3663                     isinstance(self.columns, MultiIndex)):

    ~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\indexes\base.py in __contains__(self, key)
       3918     @Appender(_index_shared_docs['contains'] % _index_doc_kwargs)
       3919     def __contains__(self, key):
    -> 3920         hash(key)
       3921         try:
       3922             return key in self._engine

    TypeError: unhashable type: 'slice'

But this chain indexing does ( which is supposedly always bad to use ):

    df_filter[:]['bruto_prihod'] = df_filter['bruto_prihod'].str.replace(',','.')

Furthermore, this doesn't work:

    df_filter['bruto_prihod'] = pd.to_numeric(df_filter['bruto_prihod'])

But this does:

df_filter[:]['bruto_prihod'] = pd.to_numeric(df_filter['bruto_prihod'])

Why is that so? Why do I have to chain indexes for my in place assignment to work?

Also, if someone is interested, this is how I grouped the transactions:

df_filter.groupby('transakcija_sifra').agg({'bruto_prihod':np.sum}).sort_values('bruto_prihod', ascending=False)

Answer 1

To avoid the warning you can use explicit copy :

df_filter = df[(df['faza'] == 'A') &
               ( (df['grupa_id'] == 'W ODIJELO') | (df['grupa_id'] == 'ODIJELO') ) &
               ( (df['vp_kupac_id']!= 591) & (df['vp_kupac_id']!= 333) & (df['vp_kupac_id']!= 332) ) &
               ( (df['brand_naziv'] == 'JOOP') | (df['brand_naziv'] == 'JOOP!') ) &
               (df['vrsta_robe']=='ROBA') &
               (df['divizija'] == 'MEN')].copy()