Geospark IllegalArgumentException：分区数必须 >= 0

问题描述

我试图在几个带有几何图形的表上运行一个简单的相交并得到这个错误。

IllegalArgumentException: Number of partitions must be >= 0

我的脚本。

tableA.
join(tableB, expr("ST_Intersects(geom, point)")).
show

这是表 A。它有几百万行。

spark.table("ta").
withColumn("point", expr("ST_Point(CAST(lon AS Decimal(24,20)), CAST(lat AS Decimal(24,20)))"))

结果。

+-----------+-----------+--------------------+
|        lat|        lon|               point|
+-----------+-----------+--------------------+
|  44.978577|  30.172431|POINT (30.172431 ...|
|  44.707343|  30.794019|POINT (30.794019 ...|
|  44.817301|  30.704576|POINT (30.704576 ...|
|  44.710767|  30.657547|POINT (30.657547 ...|
|   44.88699|  30.521111|POINT (30.521111 ...|
|     44.779|    30.6296|POINT (30.6296 55...|
|  44.653987|  30.572032|POINT (30.572032 ...|
|  44.763931|  30.601646|POINT (30.601646 ...|
|44.44440079|30.50870132|POINT (30.5087013...|
|  44.707493|  30.575095|POINT (30.575095 ...|
|  44.566665|   30.56598|POINT (30.56598 5...|
|   44.58322|  30.209977|POINT (30.209977 ...|
|  44.687525|  30.665842|POINT (30.665842 ...|
|44.90000153|30.62870026|POINT (30.6287002...|
|   44.85094|  30.560021|POINT (30.560021 ...|
|   44.83429|   30.49514|POINT (30.49514 5...|
|  44.740523|  30.890627|POINT (30.890627 ...|
|  44.544804|  30.328373|POINT (30.328373 ...|
|   44.46986|    30.5456|POINT (30.5456 55...|
|    44.8912|    30.6089|POINT (30.6089 55...|
+-----------+-----------+--------------------+

这是表 B。它只有 1 行。

spark.table("tb").
withColumn("geom", expr("ST_GeomFromWKT(wkt)"))

什么节目给我。

+--------------------+--------------------+
|                 wkt|                geom|
+--------------------+--------------------+
|MULTIPOLYGON (((3...|MULTIPOLYGON (((3...|
+--------------------+--------------------+

这个错误是怎么回事？ 我如何解决它？