databricks notebook 中的 RandomForestClassifier Explainer Dashboard output 未呈現

Question

我正在嘗試使用 ExplainerDashboard package 渲染 RandomForestClassifier model 儀表板，但它沒有在筆記本中渲染儀表板。

這是代碼

model = RandomForestClassifier(n_estimators=50, max_depth=10).fit(X_train, y_train)
explainer = ClassifierExplainer(model, X_test, y_test) 
ExplainerDashboard(explainer).run()

我低於 output

=========================================================================

Detected RandomForestClassifier model: Changing class type to RandomForestClassifierExplainer...

Note: model_output=='probability', so assuming that raw shap output of RandomForestClassifier is in probability space...

Generating self.shap_explainer = shap.TreeExplainer(model)
Building ExplainerDashboard..

Detected notebook environment, consider setting mode='external', mode='inline' or mode='jupyterlab' to keep the notebook interactive while the dashboard is running...

Warning: calculating shap interaction values can be slow! Pass shap_interaction=False to remove interactions tab.

Generating layout...

Calculating shap values...

Calculating prediction probabilities...

Calculating metrics...

Calculating confusion matrices...

Calculating classification_dfs...

Calculating roc auc curves...

Calculating pr auc curves...

Calculating liftcurve_dfs...

Calculating shap interaction values... (this may take a while)

Reminder: TreeShap computational complexity is O(TLD^2), where T is the number of trees, L is the maximum number of leaves in any tree and D the maximal depth of any tree. So reducing these will speed up the calculation.

Calculating dependencies...

Calculating permutation importances (if slow, try setting n_jobs parameter)...

Calculating pred_percentiles...

Calculating predictions...

Calculating ShadowDecTree for each individual decision tree...

Reminder: you can store the explainer (including calculated dependencies) with explainer.dump('explainer.joblib') and reload with e.g. ClassifierExplainer.from_file('explainer.joblib')

Registering callbacks...

Starting ExplainerDashboard on http://19.221.249.249:8055

Dash is running on http://0.0.0.0:8055/


 * Serving Flask app 'explainerdashboard.dashboards' (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on all addresses.
   WARNING: This is a development server. Do not use it in a production deployment.
 * Running on http://19.221.249.249:8055/

=========================================================================

但儀表板未在筆記本中呈現。 我也嘗試使用 InlineExplainer，它返回 <IPython.lib.display.IFrame at 0x7f4eea3e1c70>

您能否提出在數據塊筆記本中呈現儀表板的任何想法

Answer 1

要在筆記本中呈現儀表板，您應該使用InlineExplainer 。 有了它，您可以 plot model 性能或形狀值，例如文檔中的說明。

您可以使用以下代碼作為參考：

from sklearn.ensemble import RandomForestClassifier
from explainerdashboard.datasets import titanic_survive
from explainerdashboard import ClassifierExplainer, ExplainerDashboard
from explainerdashboard import InlineExplainer

X_train, y_train, X_test, y_test = titanic_survive()

model = RandomForestClassifier(n_estimators=50, max_depth=10).fit(X_train, y_train)
explainer = ClassifierExplainer(model, X_test, y_test) 

InlineExplainer(explainer).shap.overview()

Output：

Answer 2

請執行以下步驟您將能夠獲得 ExplainerDashboard 視圖：

步驟 1) 在 Databricks 集群中設置環境變量

例如：

DASH_REQUEST_PATHNAME_PREFIX=/driver-proxy/o/40xxxxxxxx/1004xxxxx/8888

工作區 ID：40xxxxxxx 集群 ID：1004xxxxxxxx 端口號：8888

步驟 2) 安裝 explainerdashboard 庫

%pip 安裝解釋器儀表板

步驟 3) 驗證“破折號”的示例代碼

從 sklearn.ensemble 導入 RandomForestClassifier

從 explainerdashboard 導入 ClassifierExplainer，ExplainerDashboard

從 explainerdashboard.datasets 導入 titanic_survive, feature_descriptions

X_train, y_train, X_test, y_test = titanic_survive()

model = RandomForestClassifier(n_estimators=50, max_depth=10).fit(X_train, y_train)

解釋器 = ClassifierExplainer（模型，X_test，y_test，

cats=['Deck', 'Embarked'],

描述=feature_descriptions，

labels=['未存活', '存活'])

ExplainerDashboard（解釋器，模式='破折號'，

重要性=假，

model_summary=假，

貢獻=真，

whatif=假，

shap_dependence=假，

shap_interaction=假，

決策樹=假）。運行（8888）

步驟 4) 儀表板 URL

https://xxxxxxxx.databricks.com/driver-proxy/o/ <workspaceID/clusterID/8888