[英]How to assign task to a specific CPU core?
I read material of distributed devices
in tensorflow
that training can be assigned to a specific CPU core. 我在tensorflow
中阅读了distributed devices
材料,可以将训练分配给特定的CPU核心。
Can we assign a task/thread to a CPU core to achieve concurrent or parallel processing? 我们可以将任务/线程分配给CPU内核以实现并发或并行处理吗?
with tf.device("/job:ps/task:0"):
weights_1 = tf.Variable(...)
biases_1 = tf.Variable(...)
with tf.device("/job:ps/task:1"):
weights_2 = tf.Variable(...)
biases_2 = tf.Variable(...)
You can get the current pid of the python process and use a third party utility like taskset to assign it to a CPU core. 您可以获取python进程的当前pid并使用第三方实用程序(如taskset)将其分配给CPU核心。
Dont know much about tensorflow but I think the GIL will come into Play here.You will have to use multiprocessing and assign each process to a dufferent core. 不太了解tensorflow,但我认为GIL将在这里发挥作用。你将不得不使用多处理并将每个进程分配给一个不同的核心。
You can bind a particular thread of process to an arbitrary core (assuming you are using linux). 您可以将特定的进程线程绑定到任意核心(假设您使用的是linux)。 This works not only for python but for any process. 这不仅适用于python,也适用于任何进程。 I made a python script to show how you can do that. 我制作了一个python脚本来展示你如何做到这一点。
You can get thread ids via ps
command: [user@dev ~]$ ps -Lo pid,%cpu,lwp -p {pid} Output for me: 你可以通过ps
命令获取线程ID:[user @ dev~] $ ps -Lo pid,%cpu,lwp -p {pid}输出给我:
PID %CPU LWP
28216 98.0 28216
28216 0.0 28217
28216 0.0 28218
Here 28216 is PID of the process, while you can see there are other threads running in a simple python script. 这里28216是进程的PID,而你可以看到在一个简单的python脚本中运行其他线程。
Now you can assign a thread to a particular core via taskset
现在,您可以通过taskset
将线程分配给特定核心
taskset -cp 0-5 28218
It will show the following output: 它将显示以下输出:
pid 28218's current affinity list: 0-11
pid 28218's new affinity list: 0-5
You then can observe that some threads are bound to different set of CPUs: 然后,您可以观察到某些线程绑定到不同的CPU集:
[user@host ~]$ taskset -cp 28218
pid 28218's current affinity list: 0-5
[user@host ~]$ taskset -cp 28217
pid 28217's current affinity list: 0-11
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.