为什么我在使用 python3 C-API 和 c FILE 时遇到分段错误 *

Question

当我尝试通过 FILE * 指针将打开的 python 3 文件 object 传递给 c 例程时出现分段错误。 我尝试将 python 2 代码移植到 python 3，到目前为止，我通过 C-API PyFile_AsFile 成功完成了此操作，它不再存在于 python 3 中。

最小的 c 代码将是 demo.c

    #include <stdio.h>
    int writeArray2Integer(FILE * f) {
       fprintf(f, "test write\n");
       return 0;
    }

我编译它

 gcc -DNDEBUG -g -O3 -Wall -fPIC -c demo.c -o build/demo.o
 gcc -shared build/demo.o -o build/demo.so

调用我的演示 c 库的 Python 3 代码将是

   #!/usr/bin/env python
  
   import sys
   import ctypes
  
   _lib =  ctypes.CDLL('build/demo.so')
   _write_array_2integer_c = _lib.writeArray2Integer
   _write_array_2integer_c.argtypes = [ctypes.c_void_p]
   _write_array_2integer_c.restype = ctypes.c_int
   
   def writeArray2Integer(f):
       ctypes.pythonapi.PyObject_AsFileDescriptor.argtypes = [ctypes.py_object]
       ctypes.pythonapi.PyObject_AsFileDescriptor.restype =  ctypes.c_int
       fd = ctypes.pythonapi.PyObject_AsFileDescriptor(f) # Segmentation fault here !!
       pf = ctypes.pythonapi.fdopen(fd, "w")
       out = _write_array_2integer_c(pf)

   if __name__ == "__main__":
       f = open("/tmp/toto.txt", "w")
       writeArray2Integer(f)
       f.close()

有 gdb 踪迹

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7e254a5 in __GI__IO_fwrite (buf=buf@entry=0x7ffff75c5000, size=size@entry=1, count=count@entry=11, fp=0x558bf310)
    at iofwrite.c:37
37      iofwrite.c: No such file or directory.

和 gdb 回溯完整

(gdb) bt full
#0  0x00007ffff7e254a5 in __GI__IO_fwrite (buf=buf@entry=0x7ffff75c5000, size=size@entry=1, count=count@entry=11, fp=0x558bf310)
    at iofwrite.c:37
        _IO_acquire_lock_file = <optimized out>
        request = 11
        written = 0
#1  0x00007ffff75c4141 in fprintf (__fmt=0x7ffff75c5000 "test write\n", __stream=<optimized out>)
    at /usr/include/x86_64-linux-gnu/bits/stdio2.h:100
No locals.
#2  writeArray2Integer (f=<optimized out>) at demo.c:4
No locals.
#3  0x00007ffff75dc630 in ffi_call_unix64 ()
   from /home/lahcen/Documents/thirdparty/deps/python3.7.5-linux/lib/python3.7/lib-dynload/../../libffi.so.6
No symbol table info available.
#4  0x00007ffff75dbfed in ffi_call ()
   from /home/lahcen/Documents/thirdparty/deps/python3.7.5-linux/lib/python3.7/lib-dynload/../../libffi.so.6
No symbol table info available.
#5  0x00007ffff75f201e in _call_function_pointer (argcount=1, resmem=0x7fffffffd3d0, restype=<optimized out>,
    atypes=0x7fffffffd390, avalues=0x7fffffffd3b0, pProc=0x7ffff75c4120 <writeArray2Integer>, flags=4353)
    at /usr/local/src/conda/python-3.7.5/Modules/_ctypes/callproc.c:829
        error_object = 0x0
        cc = 2
        _save = <optimized out>
        space = 0x7ffff761aab0
        cif = {abi = FFI_UNIX64, nargs = 1, arg_types = 0x7fffffffd390, rtype = 0x7ffff7636258, bytes = 0, flags = 10}
        _save = <optimized out>
        error_object = <optimized out>
        space = <optimized out>
        cif = {abi = <optimized out>, nargs = <optimized out>, arg_types = <optimized out>, rtype = <optimized out>,
          bytes = <optimized out>, flags = <optimized out>}
        cc = <optimized out>
        _py_xdecref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
        temp = <optimized out>
        temp = <optimized out>
#6  _ctypes_callproc (pProc=0x7ffff75c4120 <writeArray2Integer>, argtuple=<optimized out>, flags=4353, argtypes=<optimized out>,
    restype=0x55555594d140, checker=0x0) at /usr/local/src/conda/python-3.7.5/Modules/_ctypes/callproc.c:1186
        i = <optimized out>
        n = 1
        argcount = 1
        argtype_count = <optimized out>
        resbuf = 0x7fffffffd3d0
        args = <optimized out>
        pa = <optimized out>
        atypes = 0x7fffffffd390

Answer 1

1. 我会尝试使用fileno()方法直接从 python 文件 object 使用文件描述符。
2. 写入后我认为你应该刷新 stream，否则你不会写入 output（直到刷新仅存储在 memory 缓冲区中的 output）。

所以总结writeArray2Integer function 应该是这样的：

def writeArray2Integer(f):
   fd = f.fileno()
   pf = ctypes.pythonapi.fdopen(fd, "w")
   out = _write_array_2integer_c(pf)
   ctypes.pythonapi.fflush(pf)

为我工作 python 3;)

更新：下面的代码适用于 python 2 和 python 3。您可以省略初始化 artypes 并在 python 3 中重新键入，但没有它们 python 2 会出现分段错误（我认为这是因为它假设 88422512004-688 是整数不适用于 64 位系统上的指针）。

import ctypes
_lib =  ctypes.CDLL('build/demo.so')
_write_array_2integer_c = _lib.writeArray2Integer
_write_array_2integer_c.argtypes = [ ctypes.c_void_p ]

def writeArray2Integer(f):
    fd = f.fileno()

    fdopen = ctypes.pythonapi.fdopen
    fdopen.argtypes = [ ctypes.c_int, ctypes.c_void_p ]
    fdopen.restype = ctypes.c_void_p

    fflush = ctypes.pythonapi.fflush
    fflush.argtypes = [ ctypes.c_void_p ]
    fflush.restype = ctypes.c_int

    fp = fdopen(fd, "w")
    out = _write_array_2integer_c(fp)
    fflush(fp)

if __name__ == "__main__":
    f = open("toto.txt", "w")
    writeArray2Integer(f)
    f.close()