[英]Varnish error "Uncached req.body can only be consumed once" by extracting response header from subrequest and add it to the original request
I have an api with jwt authentication (bearer token).我有一个 api 和 jwt 身份验证(不记名令牌)。 The jwt is sent on every api request.每个 api 请求都会发送 jwt。 For validating the jwt I have a specific route in my backend ( GET /_jwt_user_sub
).为了验证 jwt,我在后端有一个特定的路由 ( GET /_jwt_user_sub
)。 A request to this route with a valid jwt returns a X-User
response header with code 200
and Content-Type: application/vnd.user-sub
.使用有效 jwt 对该路由的请求返回X-User
响应 header,代码为200
和Content-Type: application/vnd.user-sub
。 I configured Varnish to make on every api call ( GET
and POST
) a sub request to this route, extract the X-User
header from the response and add it to the originally api request.我将 Varnish 配置为在每次 api 调用( GET
和POST
)时向该路由发出一个子请求,从响应中提取X-User
header 并将其添加到最初的 api 请求中。 This jwt user id response should be cached by varnish.此 jwt 用户 ID 响应应由清漆缓存。 For api requests with GET
method this works fine but not for api calls with POST
method because Varnish by default do all backend requests with GET
.对于使用GET
方法的 api 请求,这可以正常工作,但对于使用POST
方法的 api 调用则不行,因为 Varnish 默认情况下使用GET
执行所有后端请求。 So I override this behaviour in the vcl_backend_fetch
sub routine (see following vcl configuration).所以我在vcl_backend_fetch
子例程中覆盖了这个行为(参见下面的 vcl 配置)。
With my vcl configuration I get a Error 503 Backend fetch failed
.使用我的 vcl 配置,我得到一个Error 503 Backend fetch failed
。 By debugging with varnishlog
I see that a vcl error is thrown: VCL_Error Uncached req.body can only be consumed once.
通过使用varnishlog
调试,我看到抛出了一个 vcl 错误: VCL_Error Uncached req.body can only be consumed once.
. . I am not sure how to rewrite the configuration correctly, as the error says at one point the configuration tries to consume a request body twice without caching it.我不确定如何正确重写配置,因为错误表明配置在某一时刻尝试使用请求主体两次而不缓存它。 Varnish should only cache GET
api calls, no POST
calls. Varnish 应该只缓存GET
api 调用,没有POST
调用。
This is my vcl configuration:这是我的 vcl 配置:
vcl 4.1;
import std;
import xkey;
backend app {
.host = "nginx";
.port = "8080";
}
sub vcl_recv {
set req.backend_hint = app;
// Retrieve user id and add it to the forwarded request.
call jwt_user_sub;
// Don't cache requests other than GET and HEAD.
if (req.method != "GET" && req.method != "HEAD") {
return (pass);
}
return (hash);
}
sub vcl_hit {
if (obj.ttl >= 0s) {
return (deliver);
}
if (obj.ttl + obj.grace > 0s) {
if (!std.healthy(req.backend_hint)) {
return (deliver);
} else if (req.http.cookie) {
return (miss);
}
return (deliver);
}
return (miss);
}
// Called when the requested object has been retrieved from the backend
sub vcl_backend_response {
if (bereq.http.accept ~ "application/vnd.user-sub"
&& beresp.status >= 500
) {
return (abandon);
}
}
// Sub-routine to get jwt user id
sub jwt_user_sub {
// Prevent tampering attacks on the mechanism
if (req.restarts == 0
&& (req.http.accept ~ "application/vnd.user-sub"
|| req.http.x-user
)
) {
return (synth(400));
}
if (req.restarts == 0) {
// Backup accept header, if set
if (req.http.accept) {
set req.http.x-fos-original-accept = req.http.accept;
}
set req.http.accept = "application/vnd.user-sub";
// Backup original URL
set req.http.x-fos-original-url = req.url;
set req.http.x-fos-original-method = req.method;
set req.url = "/_jwt_user_sub";
set req.method = "GET";
return (hash);
}
// Rebuild the original request which now has the hash.
if (req.restarts > 0
&& req.http.accept == "application/vnd.user-sub"
) {
set req.url = req.http.x-fos-original-url;
set req.method = req.http.x-fos-original-method;
unset req.http.x-fos-original-url;
if (req.http.x-fos-original-accept) {
set req.http.accept = req.http.x-fos-original-accept;
unset req.http.x-fos-original-accept;
} else {
// If accept header was not set in original request, remove the header here.
unset req.http.accept;
}
if (req.http.x-fos-original-method == "POST") {
return (pass);
}
return (hash);
}
}
sub vcl_backend_fetch {
if (bereq.http.x-fos-original-method == "POST") {
set bereq.method = "POST";
}
}
sub vcl_deliver {
// On receiving the hash response, copy the hash header to the original
// request and restart.
if (req.restarts == 0
&& resp.http.content-type ~ "application/vnd.user-sub"
) {
set req.http.x-user = resp.http.x-user;
return (restart);
}
}
The built-in VCL says that only GET
and HEAD
requests are cacheable by default. 内置的 VCL说默认情况下只有GET
和HEAD
请求是可缓存的。 You can override this and cache POST
calls, but unless you perform explicit caching of the request body (which is stored in req.body
), the body itself will only be processed once.您可以覆盖它并缓存POST
调用,但除非您对请求正文(存储在req.body
中)执行显式缓存,否则正文本身只会被处理一次。
Because you perform restart
calls in your VCL, the request body is processed multiple times.因为您在 VCL 中执行restart
调用,请求主体会被处理多次。 Unfortunately, at the 2nd attempt it is no longer available.不幸的是,在第二次尝试时,它不再可用。
To tackle this issue, you can use the std.std.cache_req_body(BYTES size)
function to explicitly cache the request body.要解决此问题,您可以使用std.std.cache_req_body(BYTES size)
function 显式缓存请求正文。 Most of the times this will be for POST
calls but the HTTP spec also allows you to have request bodies for GET
calls.大多数情况下,这将用于POST
调用,但 HTTP 规范还允许您拥有GET
调用的请求主体。
See https://varnish-cache.org/docs/6.0/reference/vmod_generated.html#func-cache-req-body for more information.有关详细信息,请参阅https://varnish-cache.org/docs/6.0/reference/vmod_generated.html#func-cache-req-body 。
Here's how to implement this function in your vcl_recv
subroutine:以下是如何在您的vcl_recv
子例程中实现此 function:
sub vcl_recv {
set req.backend_hint = app;
// Cache the request body for POST calls
if(req.method == "POST") {
std.cache_req_body(10K);
}
// Retrieve user id and add it to the forwarded request.
call jwt_user_sub;
// Don't cache requests other than GET and HEAD.
if (req.method != "GET" && req.method != "HEAD") {
return (pass);
}
return (hash);
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.