[英]Can I have two multithreaded functions running at the same time?
I'm very new to multi-threading. 我对多线程非常陌生。 I have 2 functions in my python script.
我的python脚本中有2个函数。 One function
enqueue_tasks
iterates through a large list of small items and performs a task on each item which involves appending an item to a list (lets call it master_list
). 一个函数
enqueue_tasks
遍历大量小项目,并对每个项目执行一项任务,该任务涉及将一个项目追加到列表中(我们称其为master_list
)。 This I already have multi-threaded using futures. 我已经使用期货使用了多线程。
executor = concurrent.futures.ThreadPoolExecutor(15) # Arbitrarily 15
futures = [executor.submit(enqueue_tasks, group) for group in grouper(key_list, 50)]
concurrent.futures.wait(futures)
I have another function process_master
that iterates through the master_list
above and checks the status of each item in the list, then does some operation. 我有另一个函数
process_master
,它遍历上面的master_list
并检查列表中每个项目的状态,然后执行一些操作。
Can I use the same method above to use multi-threading for process_master
? 我可以使用与上述相同的方法对
process_master
使用多线程吗? Furthermore, can I have it running at the same time as enqueue_tasks
? 此外,我可以让它与
enqueue_tasks
同时运行吗? What are the implications of this? 这意味着什么?
process_master
is dependent on the list from enqueue_tasks
, so will running them at the same time be a problem? process_master
依赖于enqueue_tasks
的列表,因此同时运行它们会成为问题吗? Is there a way I can delay the second function a bit? 有什么办法可以延迟第二个功能吗? (using
time.sleep
perhaps)? (也许使用
time.sleep
)?
No, this isn't safe. 不,这不安全。 If
enqueue_tasks
and process_master
are running at the same time, you could potentially be adding items to master_list
inside enqueue_tasks
at the same time process_master
is iterating over it. 如果
enqueue_tasks
和process_master
同时运行,则可能有可能在process_master
进行迭代的同时将项目添加到enqueue_tasks
中的master_list
。 Changing the size of an iterable while you iterate over it causes undefined behavior in Python, and should always be avoided. 在迭代时更改Iterable的大小会在Python中导致未定义的行为,应始终避免。 You should use a
threading.Lock
to protect the code that adds items to master_list
, as well as the code that iterates over master_list
, to ensure they never run at the same time. 您应该使用
threading.Lock
来保护将项目添加到master_list
的代码以及在master_list
上迭代的代码,以确保它们永远不会同时运行。
Better yet, use a Queue.Queue
( queue.Queue
in Python 3.x) instead of a list
, which is a thread-safe data structure. 更好的是,使用
Queue.Queue
(Python 3.x中的queue.Queue
)代替list
,后者是线程安全的数据结构。 Add items to the Queue
in enqueue_tasks
, and get
items from the Queue
in process_master
. 在
enqueue_tasks
项目添加到Queue
中,并在process_master
从Queue
中get
项目。 That way process_master
can safely run a the same time as enqueue_tasks
. 这样,
process_master
可以安全地与enqueue_tasks
同时运行。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.