如何对测试数据使用逻辑回归

Question

我在泰坦尼克号模型上使用逻辑回归，而 PyCharm 要求我只传递带有布尔值的数据帧：

Traceback (most recent call last):
  File "C:/Users/security/Downloads/AP/Titanic-Kaggle/TItanic-Kaggle.py", line 29, in <module>
    predictions = logReg.predict(test[test_data])
  File "C:\Users\security\Anaconda3\envs\TItanic-Kaggle.py\lib\site-packages\pandas\core\frame.py", line 2914, in __getitem__
    return self._getitem_frame(key)
  File "C:\Users\security\Anaconda3\envs\TItanic-Kaggle.py\lib\site-packages\pandas\core\frame.py", line 3009, in _getitem_frame
    raise ValueError('Must pass DataFrame with boolean values only')
ValueError: Must pass DataFrame with boolean values only

我不明白为什么，因为在训练模型时在逻辑回归上使用了完全相同的特征，并且当时很受欢迎。 这是我的代码（忽略代码重复。这是我将要解决的问题）：

import pandas as pd
import warnings
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

warnings.filterwarnings("ignore", category=FutureWarning)

train = pd.read_csv("https://raw.githubusercontent.com/oo92/Titanic-Kaggle/master/train.csv")
test = pd.read_csv("https://raw.githubusercontent.com/oo92/Titanic-Kaggle/master/test.csv")

train['Sex'] = train['Sex'].replace(['female', 'male'], [0, 1])
train['Embarked'] = train['Embarked'].replace(['C', 'Q', 'S'], [1, 2, 3])
train['Age'].fillna(train.groupby('Sex')['Age'].transform("median"), inplace=True)
train['HasCabin'] = train['Cabin'].notnull().astype(int)
train['Relatives'] = train['SibSp'] + train['Parch']
train_data = train[['Pclass', 'Sex', 'Relatives', 'Fare', 'Age', 'Embarked', 'HasCabin']]
x_train, x_validate, y_train, y_validate = train_test_split(train_data, train['Survived'], test_size=0.22, random_state=0)

test['Sex'] = test['Sex'].replace(['female', 'male'], [0, 1])
test['Embarked'] = test['Embarked'].replace(['C', 'Q', 'S'], [1, 2, 3])
test['Age'].fillna(test.groupby('Sex')['Age'].transform("median"), inplace=True)
test['HasCabin'] = test['Cabin'].notnull().astype(int)
test['Relatives'] = test['SibSp'] + test['Parch']
test_data = test[['Pclass', 'Sex', 'Relatives', 'Fare', 'Age', 'Embarked', 'HasCabin']]

logReg = LogisticRegression()
logReg.fit(x_train, y_train)

predictions = logReg.predict(test[test_data])
submission = pd.DataFrame({'PassengerId': test['PassengerId'], 'Survived': predictions})

filename = 'Titanic-Submission.csv'
submission.to_csv(filename, index=False)

具体来说，Python 对这个片段提出了问题：

test_data = test[['Pclass', 'Sex', 'Relatives', 'Fare', 'Age', 'Embarked', 'HasCabin']]

...

predictions = logReg.predict(test[test_data])

更新

我已将我的predictions变量更改为：

predictions = logReg.predict(test_data)

现在这是我的堆栈跟踪：

Traceback (most recent call last):
  File "C:/Users/security/Downloads/AP/Titanic-Kaggle/TItanic-Kaggle.py", line 29, in <module>
    predictions = logReg.predict(test_data)
  File "C:\Users\security\Anaconda3\envs\TItanic-Kaggle.py\lib\site-packages\sklearn\linear_model\base.py", line 281, in predict
    scores = self.decision_function(X)
  File "C:\Users\security\Anaconda3\envs\TItanic-Kaggle.py\lib\site-packages\sklearn\linear_model\base.py", line 257, in decision_function
    X = check_array(X, accept_sparse='csr')
  File "C:\Users\security\Anaconda3\envs\TItanic-Kaggle.py\lib\site-packages\sklearn\utils\validation.py", line 573, in check_array
    allow_nan=force_all_finite == 'allow-nan')
  File "C:\Users\security\Anaconda3\envs\TItanic-Kaggle.py\lib\site-packages\sklearn\utils\validation.py", line 56, in _assert_all_finite
    raise ValueError(msg_err.format(type_err, X.dtype))
ValueError: Input contains NaN, infinity or a value too large for dtype('float64').

这意味着我对测试数据的特征选择/工程没有通过

Answer 1

您在Fare列中有一个NaN值，但您不关心。 像更换Age一样更换它可以解决这个问题。 这是模型的最佳解决方案吗？ 这是一个不同的论点，但这解决了问题。

train['Fare'].fillna(train.groupby('Sex')['Age'].transform("median"), inplace=True)
test['Fare'].fillna(train.groupby('Sex')['Age'].transform("median"), inplace=True)

Answer 2

使用x_validate预测没有问题。 尝试：

>>> predictions = logReg.predict(x_validate)

所以test_data肯定有问题。 获取有关数据帧的一些信息并进行比较：

>>> x_validate.info(verbose=True)                                                                                                                                                          
<class 'pandas.core.frame.DataFrame'>
Int64Index: 197 entries, 495 to 45
Data columns (total 7 columns):
Pclass       197 non-null int64
Sex          197 non-null int64
Relatives    197 non-null int64
Fare         197 non-null float64
Age          197 non-null float64
Embarked     197 non-null int64
HasCabin     197 non-null int64
dtypes: float64(2), int64(5)
memory usage: 12.3 KB

>>> test_data.info(verbose=True)                                                                                                                                                           
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 418 entries, 0 to 417
Data columns (total 7 columns):
Pclass       418 non-null int64
Sex          418 non-null int64
Relatives    418 non-null int64
Fare         417 non-null float64
Age          418 non-null float64
Embarked     418 non-null int64
HasCabin     418 non-null int64
dtypes: float64(2), int64(5)
memory usage: 22.9 KB

看起来这里有一个 NaN：

Fare         417 non-null float64