[英]Linear Regression with sklearn using categorical variables
I am trying to run a usual linear regression in Python using sk-learn, but I have some categorical data that I don't know exactly how to handle, especially because I imported the data using pandas read.csv()
and I have learned from previous experiences and reading that Pandas and sk-learn don't get along quite well (yet). 我试图使用sk-learn在Python中运行常用的线性回归,但是我有一些我不知道如何处理的分类数据,特别是因为我使用pandas
read.csv()
导入了数据并且我学到了从以前的经验和阅读看,熊猫和sk-learn相处得不好(还)。
My data looks like this: 我的数据如下:
Salary AtBat Hits League EastDivision
475 315 81 1 0
480 479 130 0 0
500 496 141 1 1
I wanna predict Salary using AtBat, Hits, League and EastDivision, where League and EastDivision are categorical. 我想使用AtBat,Hits,League和EastDivision来预测薪水,其中League和EastDivision是绝对的。
If I import the data via numpy's loadtext()
I get a numpy array which in theory I could use with sklearn, but when I use DictVectorizer I get an error. 如果我通过numpy的
loadtext()
导入数据,我得到一个numpy数组,理论上我可以使用sklearn,但是当我使用DictVectorizer时,我得到一个错误。 My code is: 我的代码是:
import numpy as np
from sklearn.feature_extraction import DictVectorizer as DV
nphitters=np.loadtxt('Hitters.csv',delimiter=',', skiprows=1)
vec = DV( sparse = False )
catL=vec.fit_transform(nphitters[:,3:4])
And I get the error when I run the last line catL=vec.fit_transform(nphitters[:,3:4])
, the error is 当我运行最后一行
catL=vec.fit_transform(nphitters[:,3:4])
时出现错误,错误是
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/dist-packages/sklearn/feature_extraction/dict_vectorizer.py", line 142, in fit_transform
self.fit(X)
File "/usr/lib/python2.7/dist-packages/sklearn/feature_extraction/dict_vectorizer.py", line 107, in fit
for f, v in six.iteritems(x):
File "/usr/lib/python2.7/dist-packages/sklearn/externals/six.py", line 268, in iteritems
return iter(getattr(d, _iteritems)())
AttributeError: 'numpy.ndarray' object has no attribute 'iteritems'
I don't know how to fix it, and another thing is, once I get the categorical data working, how do I run the regression? 我不知道如何解决它,另一件事是,一旦我得到分类数据,我该如何运行回归? Just as if the categorical variable were another numeric variable?
就像分类变量是另一个数字变量一样?
I have found several questions similar to mine, but none of them have really worked for me. 我发现了几个类似于我的问题,但没有一个问题对我有用。
It looks like .fit_transform()
expects a dict
but .loadtxt()
create a numpy array. 看起来
.fit_transform()
需要一个dict
但.loadtxt()
创建一个numpy数组。
You can use .to_dict()
after reading your data with pandas
. 您可以使用
.to_dict()
与读取数据后pandas
。
Basically what happens is that you are passing a vector of 1 and 0 to a function that will take keys and values (like a dictionary) and create a table for you 基本上会发生的是你将1和0的向量传递给一个函数,该函数将获取键和值(如字典)并为你创建一个表
D = [{'foo': 1, 'bar': 2}, {'foo': 3, 'baz': 1}]
will become 会变成
array([[ 2., 0., 1.],
[ 0., 1., 3.]])
or 要么
|bar|baz|foo |<br>
|---|---|-----|<br>
| 2 | 0 | 1 |<br>
| 0 | 0 | 3 |<br>
read: http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.DictVectorizer.html 阅读: http : //scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.DictVectorizer.html
in your case, the data is ready for a linear regression as the features league and east division are dummies already. 在你的情况下,数据已准备好进行线性回归,因为功能联盟和东部分区已经是假人。
scikit-learn has two new functions which do this for you scikit-learn有两个新功能可以帮到你
sklearn.preprocessing.LabelBinarizer
sklearn.preprocessing.LabelEncoder
If your want to process multiple values in a single row, 如果您想在一行中处理多个值,
sklearn.preprocessing.MultiLabelBinarizer
eg: 例如:
array = [(dog, cat),(dog),(dog,fish)]
mb = MultiLabelBinarizer()
mb.fit_transform(array)
>> array([1, 0, 1, 0, 0, 0],
[0, 1, 0, 0, 1, 1],
[0, 0, 1, 1, 0, 0]])
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.