Python 在加载 GDAL package 时使用 zipfile 时出现分段错误

Question

我在尝试使用zipfile模块提取 ZIP 文件时遇到了一个奇怪的分段错误错误，同时导入了osgeo.gdal模块（不仅如此， osgeo.ogr产生相同的错误 - 也许它与使用 SWIG 的两个模块有关？ ）。

我能够将崩溃缩小到osgeo.gdal的特定导入，请参见下面的示例。 使用 GDB 运行 Python 解释器似乎表明崩溃源于 zlib 中某处的无效free()调用，但不幸的是我缺乏在那里进一步挖掘的经验。

有人猜测这里可能出了什么问题吗？ 或者也许一些关于如何最好地调试这样的问题的指针（呵呵）？

万分感谢！

编辑：经过更多调查，我发现针对 Conda 提供的 GDAL package （ https://github.com/conda-forge/gdal-feedstock/issues/365 ）报告了类似的问题。 将 GDAL 模块导入移动到导入列表末尾的建议解决方法修复了段错误，所以仍然存在：我很感激任何可以指出我调试此类问题的正确方向的答案:)

违规代码，（几乎）最小的例子

# >>> Comment this import to make the segfault disappear
from osgeo import gdal
# <<<

import io
import tempfile
import zipfile
import urllib3
import faulthandler

class GadmDownloader:
    BASE_URL_TEMPLATE = 'https://biogeo.ucdavis.edu/data/gadm3.6/gpkg/gadm36_{}_gpkg.zip'

    def __init__(self):
        self.http = urllib3.PoolManager()

    def download_country(self, country, output_dir = None):
        url = self.BASE_URL_TEMPLATE.format(country.upper())

        resp: urllib3.HTTPResponse = self.http.request('GET', url)
        if resp.status != 200:
            raise RuntimeError(f'Download failed: {resp.status}')

        faulthandler.enable()
        with zipfile.ZipFile(io.BytesIO(resp.data), 'r') as zipf:
            gpkg = [name for name in zipf.namelist()
                    if name.endswith('.gpkg')]
            zipf.extract(gpkg[0], output_dir)
        return gpkg[0]


def main():
    gdal.UseExceptions()

    dl = GadmDownloader()
    with tempfile.TemporaryDirectory() as tempdir:
        path = dl.download_country('gmb', tempdir)
        print(f'>> Path: {path}')

if __name__ == "__main__":
    main()

Output

$ python -X dev gadm.py                                                                                                                                                                                               (minikube:default)
/usr/lib/python3.8/site-packages/osgeo/__init__.py:15: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
Fatal Python error: Segmentation fault

Current thread 0x00007ffb15579740 (most recent call first):
  File "/usr/lib/python3.8/zipfile.py", line 724 in _get_decompressor
  File "/usr/lib/python3.8/zipfile.py", line 819 in __init__
  File "/usr/lib/python3.8/zipfile.py", line 1571 in open
  File "/usr/lib/python3.8/zipfile.py", line 1700 in _extract_member
  File "/usr/lib/python3.8/zipfile.py", line 1630 in extract
  File "gadm.py", line 29 in download_country
  File "gadm.py", line 38 in main
  File "gadm.py", line 42 in <module>
[1]    200128 segmentation fault (core dumped)  python -X dev -d -X showmalloc gadm.py

GDB 回溯

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7e3aa50 in free () from /usr/lib/libc.so.6
(gdb) bt
#0  0x00007ffff7e3aa50 in free () from /usr/lib/libc.so.6
#1  0x00007ffff485ae0f in inflateReset2 () from /usr/lib/libcfitsio.so.9
#2  0x00007ffff39ec1a4 in inflateInit2_ () from /usr/lib/libz.so.1
#3  0x00007ffff74a7880 in ?? () from /usr/lib/python3.8/lib-dynload/zlib.cpython-38-x86_64-linux-gnu.so
#4  0x00007ffff7b466ad in ?? () from /usr/lib/libpython3.8.so.1.0
#5  0x00007ffff7b43097 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.8.so.1.0
#6  0x00007ffff7b4f138 in _PyFunction_Vectorcall () from /usr/lib/libpython3.8.so.1.0
#7  0x00007ffff7b3e77c in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.8.so.1.0
#8  0x00007ffff7b3d0c4 in _PyEval_EvalCodeWithName () from /usr/lib/libpython3.8.so.1.0
#9  0x00007ffff7b47095 in _PyObject_FastCallDict () from /usr/lib/libpython3.8.so.1.0
#10 0x00007ffff7b59c2f in ?? () from /usr/lib/libpython3.8.so.1.0
#11 0x00007ffff7b47c90 in _PyObject_MakeTpCall () from /usr/lib/libpython3.8.so.1.0
#12 0x00007ffff7b42fc4 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.8.so.1.0
#13 0x00007ffff7b3d7e2 in _PyEval_EvalCodeWithName () from /usr/lib/libpython3.8.so.1.0
#14 0x00007ffff7b5e982 in ?? () from /usr/lib/libpython3.8.so.1.0
#15 0x00007ffff7b3f553 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.8.so.1.0
#16 0x00007ffff7b3d7e2 in _PyEval_EvalCodeWithName () from /usr/lib/libpython3.8.so.1.0
#17 0x00007ffff7b4f1cd in _PyFunction_Vectorcall () from /usr/lib/libpython3.8.so.1.0
#18 0x00007ffff7b3eb41 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.8.so.1.0
#19 0x00007ffff7b3d0c4 in _PyEval_EvalCodeWithName () from /usr/lib/libpython3.8.so.1.0
#20 0x00007ffff7b4f1cd in _PyFunction_Vectorcall () from /usr/lib/libpython3.8.so.1.0
#21 0x00007ffff7b3eb41 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.8.so.1.0
#22 0x00007ffff7b3d0c4 in _PyEval_EvalCodeWithName () from /usr/lib/libpython3.8.so.1.0
#23 0x00007ffff7b4f1cd in _PyFunction_Vectorcall () from /usr/lib/libpython3.8.so.1.0
#24 0x00007ffff7b3eb41 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.8.so.1.0
#25 0x00007ffff7b4f138 in _PyFunction_Vectorcall () from /usr/lib/libpython3.8.so.1.0
#26 0x00007ffff7b3e77c in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.8.so.1.0
#27 0x00007ffff7b3d0c4 in _PyEval_EvalCodeWithName () from /usr/lib/libpython3.8.so.1.0
#28 0x00007ffff7bee323 in PyEval_EvalCode () from /usr/lib/libpython3.8.so.1.0
#29 0x00007ffff7bf9b98 in ?? () from /usr/lib/libpython3.8.so.1.0
#30 0x00007ffff7bf3db3 in ?? () from /usr/lib/libpython3.8.so.1.0
#31 0x00007ffff7ab2a0b in PyRun_FileExFlags () from /usr/lib/libpython3.8.so.1.0
#32 0x00007ffff7ab2482 in PyRun_SimpleFileExFlags () from /usr/lib/libpython3.8.so.1.0
#33 0x00007ffff7c06a1a in Py_RunMain () from /usr/lib/libpython3.8.so.1.0
#34 0x00007ffff7be2bd9 in Py_BytesMain () from /usr/lib/libpython3.8.so.1.0
#35 0x00007ffff7dd6002 in __libc_start_main () from /usr/lib/libc.so.6
#36 0x000055555555504e in _start ()

系统事实

• Linux 5.7.10-1-MANJARO #1 SMP PREEMPT 2020 年 7 月 22 日星期三 15:23:40 UTC x86_64 GNU/Linux
• 拱门/Manjaro Linux
• Python 3.8.5
• GDAL 3.0.4

Answer 1

按字母倒序执行 Python 导入可解决此问题。