Python 性能问题：两个多边形之间的差异

Question

I am currently using Python 3.7 and I want to find the difference between a lot of polygons. With that I mean that if I have a polygon A and a polygon B I want to do the mathematical "A not B" operation. There are two possible outcomes of this operation as seen in the following illustration:

So two polygons that I subtract ("cut") from each other either give me a new polygon or are empty. All other cases can be ignored. The form of the polygon does not need to be exact for case 1. So it is acceptable if the polygon changes a bit.

For case 2 I need to know if the polygon is empty.

Furthermore polygon A and B do not have any "holes" in them so they can be described by only their outside border.

I already built a prototype that uses the difference operation of shapely to do this. I "cut" exactly as little as possible (once for every two polygons).

My code is a bit complex but it basically breaks down to this simple function:

def cut_hole(A : Polygon, B : Polygon) -> Polygon:
    """
    Cuts a "hole" into shapely polygon A
    :return: The polygon resulting of the operation A-B. Might be empty!
    """
    outer = A #not in my code, just to point out what I mean
    inner = B
    return outer.difference(inner)

Now my problem is that this is very slow! I work with roughly 15.000 operations per batch (30.000 polygons) and I takes about 10 to 15 min to calculate them all. I would really like to go down to under 5 mins.

Please keep in mind that this does not account for all the other operations. 15 min just for the difference operation. I can sort every polygon A to every polygon B in under 1 min. I just need a quick way to get the resulting polygon from those.

I did this test with an "good" computer (Intel core i7, 16 GB Ram). Neither the CPU or RAM was at its limit.

So the big question is: how can I speed this up?

Is there a way to translate the polygons into a form that is easier to handle? Or is there a "better" way to get the difference of two polygons?

Is there an alternative library that might be better? Or can I get shapely to use other hardware? If so what kind of hardware might that be?

Finally my next step would be to try and parallelize the "cutting". Is there an build-in way to do this quickly and efficiently? Because I did not find one in shapely.

Also I would be very grateful for tips on analyzing possible bottlenecks.

Addendum:

Some of the polygons seem to be rather complex. With that I mean that at average the more complex polygons contain 15.000 points. The not complex polygons less then 100 points. However usually (as in 99 %) polygon type A or type B are not complex at the same time.

Here is an example of an complex polygon in WKT

Answer 1

Taking your points in order:

• I highly doubt there is another, better-suited format/library for manipulating polygons in python than shapely, it is the reference package. You can try to simplify your geometries, but some rapid tests showed it is a slow operation as well ( p being the polygon you copypasted above):

   p2 = p.buffer(-10) # creating a 2nd polygon %timeit p.simplify(1) # 58.4 ms, from 15000 to 8000 points %timeit p.difference(p2) # 53.2 ms %timeit p.difference(p2.simplify(1)) # 127ms %timeit p.simplify(1).difference(p2) # 114ms

• Shapely uses GEOS under the hood. Maybe you can try to dig in that direction for lower-level solutions.

• There is no parallelism in shapely. However as you seem to have your 'As' and 'Bs' polygons already matched, you can parallelize the shapely operation through a threadpool or processpool (see multiprocessing package). If they are not matched, you can check it quickly through intersects (much faster than intersection or difference . If some of your polygons do not intersect, that will be a huge speedup.

• Considering the size of your data (5GB is a lot of geometries...), I don't think you can spare that much time other than with parallelization, as one difference takes ~70ms which gives ~1050s = 17 min for 15000 operations