[英]pymc3: how to model correlated intercept and slope in multilevel linear regression
In the Pymc3 example for multilevel linear regression (the example is here , with the radon
data set from Gelman et al.'s (2007)), the intercepts (for different counties) and slopes (for apartment with and without basement) each have a Normal prior. 在多级线性回归的Pymc3示例中(示例在这里 ,使用Gelman等人(2007)的
radon
数据集),拦截(针对不同的县)和斜坡(对于有和没有地下室的公寓)都有正常之前。 How can I model them together with a multivariate normal prior, so that I can examine the correlation between them? 如何将它们与多元正态先验模型一起建模,以便我可以检查它们之间的相关性?
The hierarchical model given in the example is like this: 示例中给出的层次模型如下所示:
with pm.Model() as hierarchical_model:
# Hyperpriors for group nodes
mu_a = pm.Normal('mu_a', mu=0., sd=100**2)
sigma_a = pm.HalfCauchy('sigma_a', 5)
mu_b = pm.Normal('mu_b', mu=0., sd=100**2)
sigma_b = pm.HalfCauchy('sigma_b', 5)
# Intercept for each county, distributed around group mean mu_a
# Above we just set mu and sd to a fixed value while here we
# plug in a common group distribution for all a and b (which are
# vectors of length n_counties).
a = pm.Normal('a', mu=mu_a, sd=sigma_a, shape=n_counties)
# Intercept for each county, distributed around group mean mu_a
b = pm.Normal('b', mu=mu_b, sd=sigma_b, shape=n_counties)
# Model error
eps = pm.HalfCauchy('eps', 5)
radon_est = a[county_idx] + b[county_idx] * data.floor.values
# Data likelihood
radon_like = pm.Normal('radon_like', mu=radon_est, sd=eps, observed=data.log_radon)
hierarchical_trace = pm.sample(2000)
And I'm trying to make some change to the priors 而我正试图对先辈们进行一些改变
with pm.Model() as correlation_model:
# Hyperpriors for group nodes
mu_a = pm.Normal('mu_a', mu=0., sd=100**2)
mu_b = pm.Normal('mu_b', mu=0., sd=100**2)
# here I want to model a and b together
# I borrowed some code from a multivariate normal model
# but the code does not work
sigma = pm.HalfCauchy('sigma', 5, shape=2)
C_triu = pm.LKJCorr('C_triu', n=2, p=2)
C = T.fill_diagonal(C_triu[np.zeros((2,2), 'int')], 1)
cov = pm.Deterministic('cov', T.nlinalg.matrix_dot(sigma, C, sigma))
tau = pm.Deterministic('tau', T.nlinalg.matrix_inverse(cov))
a, b = pm.MvNormal('mu', mu=(mu_a, mu_b), tau=tau,
shape=(n_counties, n_counties))
# Model error
eps = pm.HalfCauchy('eps', 5)
radon_est = a[county_idx] + b[county_idx] * data.floor.values
# Data likelihood
radon_like = pm.Normal('radon_like', mu=radon_est, sd=eps, observed=data.log_radon)
correlation_trace = pm.sample(2000)
Here is the error message I got: 这是我收到的错误消息:
File "<ipython-input-108-ce400c54cc39>", line 14, in <module>
tau = pm.Deterministic('tau', T.nlinalg.matrix_inverse(cov))
File "/home/olivier/anaconda3/lib/python3.5/site-packages/theano/gof/op.py", line 611, in __call__
node = self.make_node(*inputs, **kwargs)
File "/home/olivier/anaconda3/lib/python3.5/site-packages/theano/tensor/nlinalg.py", line 73, in make_node
assert x.ndim == 2
AssertionError
Clearly I've made some mistakes about the covariance matrix, but I'm new to pymc3
and completely new to theano
so have no idea how to fix it. 显然,我做了关于协方差矩阵一些错误,但我是新来的
pymc3
和完全新的theano
所以不知道如何解决它。 I gather this should be a rather common use case so maybe there have been some examples on it? 我认为这应该是一个相当常见的用例,所以可能有一些例子吗? I just can't find them.
我找不到他们。
The full replicable code and data can be seen on the example page (link given above). 完整的可复制代码和数据可以在示例页面上看到(上面给出的链接)。 I didn't include it here because it's too long and also I thought those familiar with
pymc3
are very likely already quite familiar with it:) 我没有把它包括在这里因为它太长了而且我认为熟悉
pymc3
人很可能已经非常熟悉了:)
You forgot to add one line when creating the covariance matrix you miss-specified the shape of the MvNormal. 在创建错误指定MvNormal的形状的协方差矩阵时,您忘记添加一行。 Your model should look something like this:
你的模型应该是这样的:
with pm.Model() as correlation_model:
mu = pm.Normal('mu', mu=0., sd=10, shape=2)
sigma = pm.HalfCauchy('sigma', 5, shape=2)
C_triu = pm.LKJCorr('C_triu', n=2, p=2)
C = tt.fill_diagonal(C_triu[np.zeros((2,2), 'int')], 1.)
sigma_diag = tt.nlinalg.diag(sigma) # this line
cov = tt.nlinalg.matrix_dot(sigma_diag, C, sigma_diag)
tau = tt.nlinalg.matrix_inverse(cov)
ab = pm.MvNormal('ab', mu=mu, tau=tau, shape=(n_counties, 2))
eps = pm.HalfCauchy('eps', 5)
radon_est = ab[:,0][county_idx] + ab[:,1][county_idx] * data.floor.values
radon_like = pm.Normal('radon_like', mu=radon_est, sd=eps, observed=data.log_radon)
trace = pm.sample(2000)
Notice that alternatively, you can evaluate the correlation of the intercept and the slope from the posterior of hierarchical_model
. 请注意,您也可以通过
hierarchical_model
的后验来评估截距和斜率的相关性。 You can use a frequentist method or build another Bayesian model, that takes as the observed data the result of hierarchical_model
. 您可以使用频率方法或构建另一个贝叶斯模型,该模型将
hierarchical_model
的结果作为观察数据。 May be this could be faster. 可能这可能会更快。
EDIT 编辑
If you want to evaluate the correlation of two variables from the posterior you can do something like. 如果你想从后验评估两个变量的相关性,你可以做类似的事情。
chain = hierarchical_trace[100:]
x_0 = chain['mu_a']
x_1 = chain['mu_b']
X = np.vstack((x_0, x_1)).T
and then you can run the following model: 然后你可以运行以下模型:
with pm.Model() as correlation:
mu = pm.Normal('mu', mu=0., sd=10, shape=2)
sigma = pm.HalfCauchy('sigma', 5, shape=2)
C_triu = pm.LKJCorr('C_triu', n=2, p=2)
C = tt.fill_diagonal(C_triu[np.zeros((2,2), 'int')], 1.)
sigma_diag = tt.nlinalg.diag(sigma)
cov = tt.nlinalg.matrix_dot(sigma_diag, C, sigma_diag)
tau = tt.nlinalg.matrix_inverse(cov)
yl = pm.MvNormal('yl', mu=mu, tau=tau, shape=(2, 2), observed=X)
trace = pm.sample(5000, pm.Metropolis())
You can replace x_0 and x_1 according to your needs. 您可以根据需要替换x_0和x_1。 For example you may want to do:
例如,您可能想要:
x_0 = np.random.normal(chain['mu_a'], chain['sigma_a'])
x_1 = np.random.normal(chain['mu_b'], chain['sigma_b'])
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.