当我将 isin 与 Dask 数据帧一起使用时抛出 NotImplementedError

Question

Let's say I have two dask data frames:

import dask.dataframe as dd 
import pandas as pd

dd_1 = dd.from_pandas(pd.DataFrame({'a': [1, 2,3], 'b': [6, 7, 8]}), npartitions=1)

dd_2 = dd.from_pandas(pd.DataFrame({'a': [1, 2, 5], 'b': [3, 7, 1]}), npartitions=1)

Now I want to filter the first one using the values of the column in the second one:

dd_1[dd_1.a.isin(dd_2.a)]

When I try to do this the following error is thrown:

NotImplementedError                       Traceback (most recent call last)
<ipython-input-38-850f035e0842> in <module>
----> 1 dd_1[dd_1.a.isin(dd_2.a)]

/usr/local/lib/python3.7/site-packages/dask/dataframe/core.py in isin(self, values)
   2113     @derived_from(pd.Series)
   2114     def isin(self, values):
-> 2115         return elemwise(M.isin, self, list(values))
   2116 
   2117     @insert_meta_param_description(pad=12)

/usr/local/lib/python3.7/site-packages/dask/dataframe/core.py in __getitem__(self, key)
   2045             graph = HighLevelGraph.from_collections(name, dsk, dependencies=[self, key])
   2046             return Series(graph, name, self._meta, self.divisions)
-> 2047         raise NotImplementedError()
   2048 
   2049     @derived_from(pd.DataFrame)

NotImplementedError: 

Any suggestion?

Answer 1

使用最新版本的dask (2.9.1)我个人的解决方法是将第二个series （在您的情况下为 dd_2.a）转换为pandas 。