如何在不具有多重共线性的熊猫中创建虚拟变量？

Question

Using Anaconda, Python 2.7.11, pandas 0.17.1, Mac OS X 10.11 (El Capitan), how do you drop a dummy variable from each column that you are making dummy variables out of to avoid multicollinearity (or the dummy variable trap) when fitting to a statistical model?

If one enters:

import pandas as pd
s = pd.Series(list('abca'))
pd.get_dummies(s)

Returned is:

   a  b  c
0  1  0  0
1  0  1  0
2  0  0  1
3  1  0  0

I want to drop a, b, or c columns to avoid multicollinearity.

Answer 1

This functionality will be added in pandas version 0.18.0 (currently 0.17.1). But if you would like this functionality sooner, then you will have to build the pandas library from source. The following instructions will show you how to do this. First, in a terminal, uninstall pandas by typing:

conda uninstall pandas

Then, navigate to site-packages, where Python stores its libraries:

cd /Users/[username]/anaconda/lib/python2.7/site-packages

where [username] is your username. The root of this path may be wherever your currently activated Python environment is located, to generalize beyond Anaconda. To reveal where your activated version of python is located, type:

which python

Enter these commands into terminal to clone into a repo where someone has added extra functionality to the pandas.get_dummies command:

git clone https://github.com/BranYang/pandas
cd pandas
python setup.py build_ext --inplace --force
python setup.py install

Then, open Python (or IPython):

ipython

and enter:

import pandas as pd
s = pd.Series(list('abca'))
pd.get_dummies(s, drop_first = True)

And this will be displayed:

   b  c
0  0  0
1  1  0
2  0  1
3  0  0

Thus, pd.get_dummies has dropped your first column and you have avoided the dummy variable trap!