[英]C code embedded python callback function
C code embedded python callback function,and put data to python queue through callback, when i get data from queue, it's very slow. C代码嵌入了python回调函数,并通过回调将数据放入python队列,当我从队列中获取数据时,它非常慢。
Example: 例:
c code like this 像这样的代码
static int wrap_func(const int a, const unsigned char *b)
{
long ret;
PyObject *arglist;
PyObject * result = NULL;
arglist = Py_BuildValue("(s#)", b, a);
result = PyEval_CallObject(my_callback, arglist);
/* evaluate result or handle exception */
ret = PyInt_AsLong(result);
if (result == NULL)
return -1;
Py_DECREF(result);
return ret;
}
void produce_data()
{
while(1){
//produce data to buffer, len is buffer length
//call callback func
wrap_func(buffer, len);
}
}
compile this c code to so like mywrap.so, and import this so in python python code like this: 将此c代码编译为mywrap.so,并将其导入到python python代码中,如下所示:
import multiprocessing
import mywarp # mywrap.so
class WorkerThread_a(threading.Thread):
def __init__(self, workQueue):
threading.Thread.__init__(self)
self.workQueue = workQueue
self.setDaemon(True)
def run(self):
while 1:
try:
recvdata = self.workQueue.get(block=False)
except Queue.Empty:
continue
#do sth use recvdata
workQueue = multiprocessing.Queue()
def callback_func(a):
if a:
workQueue.put(a)
return 0
def main():
tmp = WorkerThread_a(workQueue)
tmp.start()
mywarp.set_callback(callback_func)
mywarp.decode_audio()
main()
In python thread, i get data from queue, but i get data very slowly, but in c so, produce data and put to queue through python callback func quickly. 在python线程中,我从队列中获取数据,但是我得到的数据非常慢,但是在c中,生成数据并快速通过python回调函数进入队列。
how can i get data from queue quickly like in pure python code. 如何从纯python代码中快速获取队列中的数据。
I think what's happening is that your C code is never releasing the global interpreter lock (GIL), so your Python code never has a chance to run. 我认为正在发生的事情是你的C代码永远不会发布全局解释器锁(GIL),所以你的Python代码永远不会有机会运行。 When you're running multiple threads in Python code, they automatically swap the GIL between them and so share time equally, but this doesn't happen without your intervention in C code.
当您在Python代码中运行多个线程时,它们会自动在它们之间交换GIL,从而平均分配时间,但如果没有您对C代码的干预,就不会发生这种情况。
It should probably work a lot better if you acquire and release the GIL once per loop in your C-code (even though you don't do anything that doesn't need it). 如果您获取和你的C代码每个循环一次释放GIL(即使你什么也不做, 并不需要它)这也许应该工作好了很多。 All I've really done is added the macros
Py_BEGIN_ALLOW_THREADS
and Py_END_ALLOW_THREADS
at the start of the function. 我真正做的就是在函数开头添加宏
Py_BEGIN_ALLOW_THREADS
和Py_END_ALLOW_THREADS
。
static int wrap_func(const int a, const unsigned char *b)
{
Py_BEGIN_ALLOW_THREADS /* ADDED! */
/* This should give the chance for the main Python thread
to run, even though no code goes here */
Py_END_ALLOW_THREADS /* ADDED */
long ret;
PyObject *arglist;
PyObject * result = NULL;
arglist = Py_BuildValue("(s#)", b, a);
result = PyEval_CallObject(my_callback, arglist);
/* evaluate result or handle exception */
ret = PyInt_AsLong(result);
/* This is chang */
if (result == NULL)
return -1;
Py_DECREF(result);
return ret;
}
(I should say - this is an untested guess that I'm 90% sure is right, but I've been wrong before!) (我应该说 - 这是一个未经测试的猜测,我90%肯定是对的,但我以前错了!)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.