[英]Simple wrapping of C code with cython
I have a number of C functions, and I would like to call them from python. 我有很多C函数,我想从python中调用它们。 cython seems to be the way to go, but I can't really find an example of how exactly this is done.
cython似乎是要走的路,但我无法真正找到一个如何做到这一点的例子。 My C function looks like this:
我的C函数看起来像这样:
void calculate_daily ( char *db_name, int grid_id, int year,
double *dtmp, double *dtmn, double *dtmx,
double *dprec, double *ddtr, double *dayl,
double *dpet, double *dpar ) ;
All I want to do is to specify the first three parameters (a string and two integers), and recover 8 numpy arrays (or python lists. All the double arrays have N elements). 我想要做的就是指定前三个参数(一个字符串和两个整数),并恢复8个numpy数组(或python列表。所有双数组都有N个元素)。 My code assumes that the pointers are pointing to an already allocated chunk of memory.
我的代码假设指针指向已经分配的内存块。 Also, the produced C code ought to link to some external libraries.
此外,生成的C代码应该链接到一些外部库。
Here's a tiny but complete example of passing numpy arrays to an external C function, logically 这是一个很小但完整的例子,将numpy数组传递给外部C函数,逻辑上
fc( int N, double* a, double* b, double* z ) # z = a + b
using Cython. 使用Cython。 (This is surely well-known to those who know it well. Comments are welcome. Last change: 23 Feb 2011, for Cython 0.14.)
(对于那些了解它的人来说,这肯定是众所周知的。欢迎提出意见。最后修改:2011年2月23日,Cython 0.14。)
First read or skim Cython build and Cython with NumPy . 首先阅读或浏览Cython构建和使用NumPy进行Cython 。
2 steps: 2个步骤:
python f-setup.py build_ext --inplace
f.pyx
and fc.cpp
-> f.so
, a dynamic library f.pyx
和fc.cpp
- > f.so
,一个动态库 python test-f.py
import f
loads f.so
; import f
load f.so
; f.fpy( ... )
calls the C fc( ... )
. f.fpy( ... )
调用C fc( ... )
。 python f-setup.py
uses distutils
to run cython, compile and link: python f-setup.py
使用distutils
来运行cython,编译和链接:
cython f.pyx -> f.cpp
compile f.cpp
and fc.cpp
编译
f.cpp
和fc.cpp
link fo fc.o
-> f.so
, a dynamic lib that python import f
will load. 链接
fo fc.o
- > f.so
,python import f
将加载的动态库。
For students, I'd suggest: make a diagram of these steps, look through the files below, then download and run them. 对于学生,我建议:制作这些步骤的图表,查看下面的文件,然后下载并运行它们。
( distutils
is a huge, convoluted package used to make Python packages for distribution, and install them. Here we're using just a small part of it to compile and link to f.so
. This step has nothing to do with Cython, but it can be confusing; simple mistakes in a .pyx can cause pages of obscure error messages from g++ compile and link. See also distutils doc and/or SO questions on distutils .) (
distutils
是一个巨大的,错综复杂的包,用于制作Python包以进行分发,并安装它们。这里我们只使用它的一小部分来编译和链接到f.so
这一步与Cython无关,但是它可能令人困惑; .pyx中的简单错误可能导致来自g ++编译和链接的页面模糊错误消息。另请参阅distutils doc和/或有关distutils的SO问题 。)
Like make
, setup.py
will rerun cython f.pyx
and g++ -c ... f.cpp
if f.pyx
is newer than f.cpp
. 与
make
一样,如果f.pyx
比f.cpp
更新, setup.py
将重新运行cython f.pyx
和g++ -c ... f.cpp
f.cpp
。
To cleanup, rm -r build/
. 要清理,
rm -r build/
。
An alternative to setup.py
would be to run the steps separately, in a script or Makefile: setup.py
的替代方法是在脚本或Makefile中单独运行这些步骤:
cython --cplus f.pyx -> f.cpp # see cython -h
g++ -c ... f.cpp -> fo
g++ -c ... fc.cpp -> fc.o
cc-lib fo fc.o -> dynamic library f.so
. cc-lib fo fc.o -> dynamic library f.so
Modify the cc-lib-mac
wrapper below for your platform and installation: it's not pretty, but small. 为您的平台和安装修改下面的
cc-lib-mac
包装器:它不漂亮,但很小。
For real examples of Cython wrapping C, look at .pyx files in just about any SciKit . 有关Cython包装C的真实示例,请查看几乎任何SciKit中的 .pyx文件。
See also: Cython for NumPy users and SO questions/tagged/cython . 另请参阅: NumPy用户的Cython和SO问题/标记/ cython 。
To unpack the following files, cut-paste the lot to one big file, say cython-numpy-c-demo
, then in Unix (in a clean new directory) run sh cython-numpy-c-demo
. 要解压缩以下文件,将批次剪切粘贴到一个大文件,比如
cython-numpy-c-demo
,然后在Unix(在一个干净的新目录中)运行sh cython-numpy-c-demo
。
#--------------------------------------------------------------------------------
cat >f.pyx <<\!
# f.pyx: numpy arrays -> extern from "fc.h"
# 3 steps:
# cython f.pyx -> f.c
# link: python f-setup.py build_ext --inplace -> f.so, a dynamic library
# py test-f.py: import f gets f.so, f.fpy below calls fc()
import numpy as np
cimport numpy as np
cdef extern from "fc.h":
int fc( int N, double* a, double* b, double* z ) # z = a + b
def fpy( N,
np.ndarray[np.double_t,ndim=1] A,
np.ndarray[np.double_t,ndim=1] B,
np.ndarray[np.double_t,ndim=1] Z ):
""" wrap np arrays to fc( a.data ... ) """
assert N <= len(A) == len(B) == len(Z)
fcret = fc( N, <double*> A.data, <double*> B.data, <double*> Z.data )
# fcret = fc( N, A.data, B.data, Z.data ) grr char*
return fcret
!
#--------------------------------------------------------------------------------
cat >fc.h <<\!
// fc.h: numpy arrays from cython , double*
int fc( int N, const double a[], const double b[], double z[] );
!
#--------------------------------------------------------------------------------
cat >fc.cpp <<\!
// fc.cpp: z = a + b, numpy arrays from cython
#include "fc.h"
#include <stdio.h>
int fc( int N, const double a[], const double b[], double z[] )
{
printf( "fc: N=%d a[0]=%f b[0]=%f \n", N, a[0], b[0] );
for( int j = 0; j < N; j ++ ){
z[j] = a[j] + b[j];
}
return N;
}
!
#--------------------------------------------------------------------------------
cat >f-setup.py <<\!
# python f-setup.py build_ext --inplace
# cython f.pyx -> f.cpp
# g++ -c f.cpp -> f.o
# g++ -c fc.cpp -> fc.o
# link f.o fc.o -> f.so
# distutils uses the Makefile distutils.sysconfig.get_makefile_filename()
# for compiling and linking: a sea of options.
# http://docs.python.org/distutils/introduction.html
# http://docs.python.org/distutils/apiref.html 20 pages ...
# https://stackoverflow.com/questions/tagged/distutils+python
import numpy
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
# from Cython.Build import cythonize
ext_modules = [Extension(
name="f",
sources=["f.pyx", "fc.cpp"],
# extra_objects=["fc.o"], # if you compile fc.cpp separately
include_dirs = [numpy.get_include()], # .../site-packages/numpy/core/include
language="c++",
# libraries=
# extra_compile_args = "...".split(),
# extra_link_args = "...".split()
)]
setup(
name = 'f',
cmdclass = {'build_ext': build_ext},
ext_modules = ext_modules,
# ext_modules = cythonize(ext_modules) ? not in 0.14.1
# version=
# description=
# author=
# author_email=
)
# test: import f
!
#--------------------------------------------------------------------------------
cat >test-f.py <<\!
#!/usr/bin/env python
# test-f.py
import numpy as np
import f # loads f.so from cc-lib: f.pyx -> f.c + fc.o -> f.so
N = 3
a = np.arange( N, dtype=np.float64 )
b = np.arange( N, dtype=np.float64 )
z = np.ones( N, dtype=np.float64 ) * np.NaN
fret = f.fpy( N, a, b, z )
print "fpy -> fc z:", z
!
#--------------------------------------------------------------------------------
cat >cc-lib-mac <<\!
#!/bin/sh
me=${0##*/}
case $1 in
"" )
set -- f.cpp fc.cpp ;; # default: g++ these
-h* | --h* )
echo "
$me [g++ flags] xx.c yy.cpp zz.o ...
compiles .c .cpp .o files to a dynamic lib xx.so
"
exit 1
esac
# Logically this is simple, compile and link,
# but platform-dependent, layers upon layers, gloom, doom
base=${1%.c*}
base=${base%.o}
set -x
g++ -dynamic -arch ppc \
-bundle -undefined dynamic_lookup \
-fno-strict-aliasing -fPIC -fno-common -DNDEBUG `# -g` -fwrapv \
-isysroot /Developer/SDKs/MacOSX10.4u.sdk \
-I/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 \
-I${Pysite?}/numpy/core/include \
-O2 -Wall \
"$@" \
-o $base.so
# undefs: nm -gpv $base.so | egrep '^ *U _+[^P]'
!
# 23 Feb 2011 13:38
The following Cython code from http://article.gmane.org/gmane.comp.python.cython.user/5625 doesn't require explicit casts and also handles non-continous arrays: 来自http://article.gmane.org/gmane.comp.python.cython.user/5625的以下Cython代码不需要显式强制转换,也可以处理非连续数组:
def fpy(A):
cdef np.ndarray[np.double_t, ndim=2, mode="c"] A_c
A_c = np.ascontiguousarray(A, dtype=np.double)
fc(&A_c[0,0])
Basically you can write your Cython function such that it allocates the arrays (make sure you cimport numpy as np
): 基本上你可以编写你的Cython函数,以便它分配数组(确保你
cimport numpy as np
):
cdef np.ndarray[np.double_t, ndim=1] rr = np.zeros((N,), dtype=np.double)
then pass in the .data
pointer of each to your C function. 然后将每个的
.data
指针传递给你的C函数。 That should work. 这应该工作。 If you don't need to start with zeros you could use
np.empty
for a small speed boost. 如果您不需要以零开头,则可以使用
np.empty
进行小幅提升。
See the Cython for NumPy Users tutorial in the docs (fixed it to the correct link). 请参阅文档中的Cython for NumPy Users教程(将其修复为正确的链接)。
你应该看看Ctypes它可能是最容易使用的东西,如果你想要的只是一个功能。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.