Unable to fully install and import Modin Package

Question

I am trying to use the modin package to speed up my pandas dataframe calculations. In short, the installation has not been as straightforward as pip install modin

When simply running pip install modin everything seems to be going fine (except for the pip upgrade warnings). All good so far...

WARNING: You are using pip version 19.3; however, version 19.3.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.

(base) C:\Users\Merv Merzoug>pip install modin
Requirement already satisfied: modin in c:\users\merv merzoug\anaconda3\lib\site-packages (0.6.2)
Requirement already satisfied: pandas==0.25.1 in c:\users\merv merzoug\anaconda3\lib\site-packages (from modin) (0.25.1)
Requirement already satisfied: pytz>=2017.2 in c:\users\merv merzoug\anaconda3\lib\site-packages (from pandas==0.25.1->modin) (2019.3)
Requirement already satisfied: python-dateutil>=2.6.1 in c:\users\merv merzoug\anaconda3\lib\site-packages (from pandas==0.25.1->modin) (2.7.3)
Requirement already satisfied: numpy>=1.13.3 in c:\users\merv merzoug\appdata\roaming\python\python36\site-packages (from pandas==0.25.1->modin) (1.16.4)
Requirement already satisfied: six>=1.5 in c:\users\merv merzoug\anaconda3\lib\site-packages (from python-dateutil>=2.6.1->pandas==0.25.1->modin) (1.12.0)
WARNING: You are using pip version 19.3; however, version 19.3.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.

Then I try to merely import the package: import modin.pandas as pd as per the docs and I get the following traceback:

ImportError: Please `pip install modin[dask] to install compatible Dask version.

Okay... so I do what they tell me to. Run pip install modin[dask] and I receive the following...

    (base) C:\Users\Merv Merzoug>pip install modin[dask]
Requirement already satisfied: modin[dask] in c:\users\merv merzoug\anaconda3\lib\site-packages (0.6.2)
Requirement already satisfied: pandas==0.25.1 in c:\users\merv merzoug\anaconda3\lib\site-packages (from modin[dask]) (0.25.1)
Requirement already satisfied: dask>=2.1.0; extra == "dask" in c:\users\merv merzoug\anaconda3\lib\site-packages (from modin[dask]) (2.7.0)
Requirement already satisfied: distributed>=2.3.2; extra == "dask" in c:\users\merv merzoug\anaconda3\lib\site-packages (from modin[dask]) (2.7.0)
Requirement already satisfied: python-dateutil>=2.6.1 in c:\users\merv merzoug\anaconda3\lib\site-packages (from pandas==0.25.1->modin[dask]) (2.7.3)
Requirement already satisfied: pytz>=2017.2 in c:\users\merv merzoug\anaconda3\lib\site-packages (from pandas==0.25.1->modin[dask]) (2019.3)
Requirement already satisfied: numpy>=1.13.3 in c:\users\merv merzoug\appdata\roaming\python\python36\site-packages (from pandas==0.25.1->modin[dask]) (1.16.4)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in c:\users\merv merzoug\appdata\roaming\python\python36\site-packages (from distributed>=2.3.2; extra == "dask"->modin[dask]) (1.5.9)
Requirement already satisfied: tornado>=5 in c:\users\merv merzoug\anaconda3\lib\site-packages (from distributed>=2.3.2; extra == "dask"->modin[dask]) (5.1.1)
Requirement already satisfied: zict>=0.1.3 in c:\users\merv merzoug\anaconda3\lib\site-packages (from distributed>=2.3.2; extra == "dask"->modin[dask]) (0.1.3)
Requirement already satisfied: msgpack in c:\users\merv merzoug\anaconda3\lib\site-packages (from distributed>=2.3.2; extra == "dask"->modin[dask]) (0.6.2)
Requirement already satisfied: psutil>=5.0 in c:\users\merv merzoug\anaconda3\lib\site-packages (from distributed>=2.3.2; extra == "dask"->modin[dask]) (5.4.5)
Requirement already satisfied: cloudpickle>=0.2.2 in c:\users\merv merzoug\anaconda3\lib\site-packages (from distributed>=2.3.2; extra == "dask"->modin[dask]) (0.5.3)
Requirement already satisfied: click>=6.6 in c:\users\merv merzoug\anaconda3\lib\site-packages (from distributed>=2.3.2; extra == "dask"->modin[dask]) (6.7)
Requirement already satisfied: pyyaml in c:\users\merv merzoug\anaconda3\lib\site-packages (from distributed>=2.3.2; extra == "dask"->modin[dask]) (5.1.2)
Requirement already satisfied: tblib in c:\users\merv merzoug\anaconda3\lib\site-packages (from distributed>=2.3.2; extra == "dask"->modin[dask]) (1.3.2)
Requirement already satisfied: toolz>=0.7.4 in c:\users\merv merzoug\anaconda3\lib\site-packages (from distributed>=2.3.2; extra == "dask"->modin[dask]) (0.9.0)
Requirement already satisfied: six>=1.5 in c:\users\merv merzoug\anaconda3\lib\site-packages (from python-dateutil>=2.6.1->pandas==0.25.1->modin[dask]) (1.12.0)
Requirement already satisfied: heapdict in c:\users\merv merzoug\anaconda3\lib\site-packages (from zict>=0.1.3->distributed>=2.3.2; extra == "dask"->modin[dask]) (1.0.0)
WARNING: You are using pip version 19.3; however, version 19.3.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.

Okay, fine, looks like I already have all that installed... let's try to import again...

import modin.pandas as pd

and that yields the same traceback:

ImportError: Please `pip install modin[dask] to install compatible Dask version.

What am I doing incorrectly? Thanks!

Answer 1

Before the import of modin you have to define the Compute Engine.

Try this (as stated in the github project page of modin):

import os

#USE ONLY ONE OF THESE:

os.environ["MODIN_ENGINE"] = "ray"  # Modin will use Ray
os.environ["MODIN_ENGINE"] = "dask"  # Modin will use Dask

import modin.pandas as pd

Answer 2

If you are running it in colab, try using the following commands:

!pip install -U ipykernel
!pip install modin[dask]

If you are getting the same error while importing after running the above command, try restarting the kernel and importing again.

More info can be found here .