python pandas 0.16：SettingWithCopyWarning错误地报告

Question

As per my other question: Python Anaconda: how to test if updated libraries are compatible with my existing code?

I curse the day I was forced to upgrade to pandas 0.16. One of the things I don't understand is why I get a chained assignment warning when I do something as banal as adding a new field to an existing dataframe and initialising it with 1:

mydataframe['x']=1

causes the following warning:

SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead

See the the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy mydataframe['x']=1

I understand there can be problems when assigning values to a copy of a dataframe, but here I am just adding a new field to a dataframe! How am I supposed to change my code (which worked perfectly in previous versions of pandas)?

Answer 1

Here's an attempt at an answer, or at least an attempt to reproduce the message. (Note that you may only get this message once and might need to start a new shell or do %reset in ipython to get this message.)

In [1]: %reset

Once deleted, variables cannot be recovered. Proceed (y/[n])? y

In [2]: import pandas as pd

In [3]: pd.__version__
Out[3]: '0.16.0'

Here are 3 variations of setting a new column to '1'. The first two do not generate the warning, but the third one does. (Second one thanks to @Jeff's suggestion)

In [4]: df = pd.DataFrame({ 'x':[1,2,3], 'y':[77,88,99] })
   ...: df['z'] = 1

In [5]: df = pd.DataFrame({ 'x':[1,2,3], 'y':[77,88,99] })
   ...: df = df[1:]
   ...: df['z'] = 1

In [6]: df = pd.DataFrame({ 'x':[1,2,3], 'y':[77,88,99] })
   ...: df2 = df[1:]
   ...: df2['z'] = 1

-c:3: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable
/indexing.html#indexing-view-versus-copy

Perhaps others can correct me if I'm wrong, but I believe the error message here is relating to df2 being a copy of a slice of df . However, that's not really an issue as the resulting df and df2 are what I would have expected:

In [7]: df
Out[7]: 
   x   y
0  1  77
1  2  88
2  3  99

In [8]: df2
Out[8]: 
   x   y  z
1  2  88  1
2  3  99  1

I know this is going to be terrible to say, but when I get that message I just check to see whether the command did what I wanted or not and don't overly think about the warning. But whether you get a warning message or not, checking that a command did what you expected is really something you need to do all the time in pandas (or matlab, or R, or SAS, or Stata, ... )

Answer 2

This will not generate the warning:

df = pd.DataFrame({ 'x':[1,2,3], 'y':[77,88,99] })
df2 = df[1:].copy()
df2['z'] = 1