[英]How to control number of processes in GaussianMixture
There are no n_jobs
parameter for GaussianMixture . GaussianMixture没有n_jobs
参数。 Meanwhile, whenever I fit the model 同时,只要我适合模型
from sklearn.mixture import GaussianMixture as GMM
gmm = GMM(n_components=4,
init_params='random',
covariance_type='full',
tol=1e-2,
max_iter=100,
n_init=1)
gmm.fit(X, y)
it spans 16 processes and uses full CPU power of my 16 CPUs machine. 它跨越16个进程,并使用了我的16个CPU机器的全部CPU能力。 I do not want for it to be doing that. 我不希望它那样做。
In comparison, Kmeans has n_jobs
parameter that controls mutliprocessing when having multiple initializations ( n_init
> 1). 相比之下, Kmeans具有n_jobs
参数,该参数在进行多次初始化( n_init
> 1)时控制多重处理。 Here multiprocessing comes out of the blue. 在这里,多处理突如其来。
My question is where its coming from and how to control it? 我的问题是它的来源以及如何控制它?
You are observing parallel-processing in terms of basic algebraic operations, speed up by BLAS / LAPACK . 您正在观察基本代数运算方面的并行处理,并通过BLAS / LAPACK加快了处理速度。
Modifying this is not as simple as setting a n_jobs
parameter and depends on your implementation in use! 修改它并不像设置n_jobs
参数那么简单,它取决于您所使用的实现!
Common candidates are ATLAS, OpenBLAS and Intel's MKL. 常见的候选人是ATLAS,OpenBLAS和英特尔的MKL。
I recommend checking which one is used first, then act accordingly: 我建议先检查使用哪个,然后采取相应措施:
import numpy as np
np.__config__.show()
Sadly these things can get tricky . 可悲的是,这些事情可能会变得棘手 。 A valid environment for MKL for example can look like this (source) : 例如,MKL的有效环境可能如下所示(源) :
export MKL_NUM_THREADS="2"
export MKL_DOMAIN_NUM_THREADS="MKL_BLAS=2"
export OMP_NUM_THREADS="1"
export MKL_DYNAMIC="FALSE"
export OMP_DYNAMIC="FALSE"
For ATLAS, it seems, you define this at compile-time . 对于ATLAS来说,似乎是在编译时定义的。
And according to this answer , the same applies to OpenBLAS. 根据这个答案 ,同样适用于OpenBLAS。
As OP tested, it seems you can get away with setting environment-variables for OpenMP , effecting in modification of behaviour even for the open-source candidates Atlas and OpenBLAS (where a compile-time limit is the alternative): 经过OP的测试,看来您可以为OpenMP设置环境变量了,甚至对开源候选Atlas和OpenBLAS(在其中可以选择编译时)进行行为修改的情况下:
export OMP_NUM_THREADS="4";
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.