[英]pandas version is not updated after installing a new version on databricks
當我在數據塊上運行 python3.7 代碼時,我試圖解決熊貓的問題。
錯誤是:
ImportError: cannot import name 'roperator' from 'pandas.core.ops' (/databricks/python/lib/python3.7/site-packages/pandas/core/ops.py)
熊貓版本:
pd.__version__
0.24.2
我跑
from pandas.core.ops import roperator
在我的筆記本電腦上
pandas 0.25.1
所以,我嘗試在數據塊上升級熊貓。
%sh pip uninstall -y pandas
Successfully uninstalled pandas-1.1.2
%sh pip install pandas==0.25.1
Collecting pandas==0.25.1
Downloading pandas-0.25.1-cp37-cp37m-manylinux1_x86_64.whl (10.4 MB)
Requirement already satisfied: python-dateutil>=2.6.1 in /databricks/conda/envs/databricks-ml/lib/python3.7/site-packages (from pandas==0.25.1) (2.8.0)
Requirement already satisfied: numpy>=1.13.3 in /databricks/conda/envs/databricks-ml/lib/python3.7/site-packages (from pandas==0.25.1) (1.16.2)
Requirement already satisfied: pytz>=2017.2 in /databricks/conda/envs/databricks-ml/lib/python3.7/site-packages (from pandas==0.25.1) (2018.9)
Requirement already satisfied: six>=1.5 in /databricks/conda/envs/databricks-ml/lib/python3.7/site-packages (from python-dateutil>=2.6.1->pandas==0.25.1) (1.12.0)
Installing collected packages: pandas
ERROR: After October 2020 you may experience errors when installing or updating packages.
This is because pip will change the way that it resolves dependency conflicts.
We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.
mlflow 1.8.0 requires alembic, which is not installed.
mlflow 1.8.0 requires prometheus-flask-exporter, which is not installed.
mlflow 1.8.0 requires sqlalchemy<=1.3.13, which is not installed.
sklearn-pandas 2.0.1 requires numpy>=1.18.1, but you'll have numpy 1.16.2 which is incompatible.
sklearn-pandas 2.0.1 requires pandas>=1.0.5, but you'll have pandas 0.25.1 which is incompatible.
sklearn-pandas 2.0.1 requires scikit-learn>=0.23.0, but you'll have scikit-learn 0.20.3 which is incompatible.
sklearn-pandas 2.0.1 requires scipy>=1.4.1, but you'll have scipy 1.2.1 which is incompatible.
Successfully installed pandas-0.25.1
當我運行時:
import pandas as pd
pd.__version__
它還是:
0.24.2
我錯過了什么嗎?
謝謝
真的建議通過集群初始化腳本安裝庫。 %sh
命令僅在驅動程序節點上執行,而不在執行程序節點上執行。 並且它也不會影響已經運行的 Python 實例。
正確的解決方案是使用dbutils.library
commands ,如下所示:
dbutils.library.installPyPI("pandas", "1.0.1")
dbutils.library.restartPython()
這會將庫安裝到所有位置,但需要重新啟動 Python 以獲取新庫。
此外,雖然可以只指定包名,但建議明確指定版本,因為某些庫版本可能與運行時不兼容。 此外,請考慮使用已更新庫版本的較新運行時 - 檢查運行時的發行說明以找出開箱即用的庫版本。
對於較新的 Databricks 運行時,您可以使用新的魔法命令: %pip
和%conda
來安裝依賴項。 有關更多詳細信息,請參閱文檔。
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.