[英]Encoding::UndefinedConversionError when calling Chef update command from a Rails application
I'm building a Rails app that uses Chef-DK underthehood.我正在构建一个使用 Chef-DK 的 Rails 应用程序。
What I'm trying to achieve is to execute the same code as the chef update <path/to/policyfile>
.我想要实现的是执行与
chef update <path/to/policyfile>
相同的代码。
Here is the interesting part of the code :这是代码中有趣的部分:
require 'chef-dk/command/update'
chef_update = ChefDK::Command::Update.new
update_options = [policyfile_path]
update_options << '--debug' if Rails.env.development?
update = chef_update.run(update_options)
When this run, it start to update the policyfile but fails with the following output:运行时,它开始更新策略文件,但失败并显示以下输出:
Attributes already up to date
Building policy k8sworker
Expanded run list: recipe[fail2ban], recipe[os-hardening], recipe[rkhunter], recipe[ssh-hardening], recipe[ssh-keys], recipe[tincvpn]
Caching Cookbooks...
Installing tincvpn ~> 0.1.10 from github
Installing fail2ban 6.0.0
Error: Failed to generate Policyfile.lock
Reason: (Encoding::UndefinedConversionError) "\x8B" from ASCII-8BIT to UTF-8
/usr/local/lib/ruby/2.6.0/delegate.rb:349:in `write'
/usr/local/lib/ruby/2.6.0/delegate.rb:349:in `block in delegating_block'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:528:in `block in stream_to_tempfile'
/usr/local/lib/ruby/2.6.0/net/protocol.rb:497:in `call_block'
/usr/local/lib/ruby/2.6.0/net/protocol.rb:488:in `<<'
/usr/local/lib/ruby/2.6.0/net/protocol.rb:163:in `read'
/usr/local/lib/ruby/2.6.0/net/http/response.rb:293:in `block in read_body_0'
/usr/local/lib/ruby/2.6.0/net/http/response.rb:253:in `inflater'
/usr/local/lib/ruby/2.6.0/net/http/response.rb:283:in `read_body_0'
/usr/local/lib/ruby/2.6.0/net/http/response.rb:204:in `read_body'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:527:in `stream_to_tempfile'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:230:in `block in streaming_request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http/basic_client.rb:92:in `block in request'
/usr/local/lib/ruby/2.6.0/net/http.rb:1518:in `block in transport_request'
/usr/local/lib/ruby/2.6.0/net/http/response.rb:165:in `reading_body'
/usr/local/lib/ruby/2.6.0/net/http.rb:1517:in `transport_request'
/usr/local/lib/ruby/2.6.0/net/http.rb:1479:in `request'
/usr/local/lib/ruby/2.6.0/net/http.rb:1472:in `block in request'
/usr/local/lib/ruby/2.6.0/net/http.rb:920:in `start'
/usr/local/lib/ruby/2.6.0/net/http.rb:1470:in `request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http/basic_client.rb:69:in `request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:370:in `block in send_http_request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:411:in `block in retrying_http_errors'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:409:in `loop'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:409:in `retrying_http_errors'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:365:in `send_http_request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:388:in `block (2 levels) in send_http_request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:485:in `follow_redirect'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:383:in `block in send_http_request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:411:in `block in retrying_http_errors'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:409:in `loop'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:409:in `retrying_http_errors'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:365:in `send_http_request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:228:in `streaming_request'
/usr/local/bundle/gems/cookbook-omnifetch-0.8.1/lib/cookbook-omnifetch/artifactserver.rb:42:in `install'
/usr/local/bundle/gems/chef-dk-4.0.60/lib/chef-dk/policyfile/cookbook_location_specification.rb:81:in `ensure_cached'
/usr/local/bundle/gems/chef-dk-4.0.60/lib/chef-dk/policyfile_compiler.rb:181:in `block in install'
/usr/local/bundle/gems/chef-dk-4.0.60/lib/chef-dk/policyfile_compiler.rb:176:in `each'
/usr/local/bundle/gems/chef-dk-4.0.60/lib/chef-dk/policyfile_compiler.rb:176:in `install'
/usr/local/bundle/gems/chef-dk-4.0.60/lib/chef-dk/policyfile_services/install.rb:101:in `generate_lock_and_install'
/usr/local/bundle/gems/chef-dk-4.0.60/lib/chef-dk/policyfile_services/install.rb:63:in `run'
/usr/local/bundle/gems/chef-dk-4.0.60/lib/chef-dk/command/update.rb:93:in `run'
/application/app/interactors/node_actions/run_chef_update.rb:24:in `call'
...
And here is the Chef output with the trace
log level:这是带有
trace
日志级别的 Chef 输出:
worker_1 | [2019-07-03T08:02:17+00:00] TRACE: Chef::HTTP calling Chef::HTTP::Decompressor#handle_request
worker_1 | [2019-07-03T08:02:17+00:00] TRACE: Chef::HTTP calling Chef::HTTP::CookieManager#handle_request
worker_1 | [2019-07-03T08:02:17+00:00] TRACE: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_request
worker_1 | [2019-07-03T08:02:17+00:00] TRACE: Initiating GET to https://supermarket.chef.io/universe
worker_1 | [2019-07-03T08:02:17+00:00] TRACE: ---- HTTP Request Header Data: ----
worker_1 | [2019-07-03T08:02:17+00:00] TRACE: Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
worker_1 | [2019-07-03T08:02:17+00:00] TRACE: ---- End HTTP Request Header Data ----
worker_1 | [2019-07-03T08:02:20+00:00] TRACE: ---- HTTP Status and Header Data: ----
worker_1 | [2019-07-03T08:02:20+00:00] TRACE: HTTP 1.1 200 OK
worker_1 | [2019-07-03T08:02:20+00:00] TRACE: cache-control: max-age=0, private, must-revalidate
worker_1 | [2019-07-03T08:02:20+00:00] TRACE: content-encoding: gzip
worker_1 | [2019-07-03T08:02:20+00:00] TRACE: content-type: application/json; charset=utf-8
worker_1 | [2019-07-03T08:02:20+00:00] TRACE: date: Wed, 03 Jul 2019 08:02:19 GMT
worker_1 | [2019-07-03T08:02:20+00:00] TRACE: etag: W/"461b5134d1467f6950f82af1510078a0"
worker_1 | [2019-07-03T08:02:20+00:00] TRACE: server: nginx
worker_1 | [2019-07-03T08:02:20+00:00] TRACE: x-content-type-options: nosniff
worker_1 | [2019-07-03T08:02:20+00:00] TRACE: x-frame-options: SAMEORIGIN
worker_1 | [2019-07-03T08:02:20+00:00] TRACE: x-request-id: 0bc2cc4a-0399-4953-93db-699176e11e36
worker_1 | [2019-07-03T08:02:20+00:00] TRACE: x-runtime: 0.301233
worker_1 | [2019-07-03T08:02:20+00:00] TRACE: x-xss-protection: 1; mode=block
worker_1 | [2019-07-03T08:02:20+00:00] TRACE: transfer-encoding: chunked
worker_1 | [2019-07-03T08:02:20+00:00] TRACE: connection: Close
worker_1 | [2019-07-03T08:02:20+00:00] TRACE: ---- End HTTP Status/Header Data ----
worker_1 | [2019-07-03T08:02:22+00:00] TRACE: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_response
worker_1 | [2019-07-03T08:02:22+00:00] TRACE: HTTP server did not include a Content-Length header in response, cannot identify truncated downloads.
worker_1 | [2019-07-03T08:02:22+00:00] TRACE: Chef::HTTP calling Chef::HTTP::CookieManager#handle_response
worker_1 | [2019-07-03T08:02:22+00:00] TRACE: Chef::HTTP calling Chef::HTTP::Decompressor#handle_response
worker_1 | [2019-07-03T08:02:22+00:00] TRACE: Decompressing gzip response
worker_1 | [2019-07-03T08:02:23+00:00] TRACE: Chef::HTTP calling Chef::HTTP::Decompressor#handle_request
worker_1 | [2019-07-03T08:02:23+00:00] TRACE: Chef::HTTP calling Chef::HTTP::CookieManager#handle_request
worker_1 | [2019-07-03T08:02:23+00:00] TRACE: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_request
worker_1 | [2019-07-03T08:02:23+00:00] TRACE: Initiating GET to https://supermarket.chef.io/api/v1/cookbooks/fail2ban/versions/6.0.0/download
worker_1 | [2019-07-03T08:02:23+00:00] TRACE: ---- HTTP Request Header Data: ----
worker_1 | [2019-07-03T08:02:23+00:00] TRACE: Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
worker_1 | [2019-07-03T08:02:23+00:00] TRACE: ---- End HTTP Request Header Data ----
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: ---- HTTP Status and Header Data: ----
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: HTTP 1.1 302 Found
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: cache-control: no-cache
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: content-type: text/html; charset=utf-8
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: date: Wed, 03 Jul 2019 08:02:25 GMT
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: location: https://s3.amazonaws.com/community-files.opscode.com/cookbook_versions/tarballs/28864/original/fail2ban20190508-41006-1kydj0z.tar.gz?1557352785
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: server: nginx
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: x-content-type-options: nosniff
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: x-frame-options: SAMEORIGIN
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: x-request-id: 17a0d07d-cf03-45b3-956f-8eb7fcabd2a7
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: x-runtime: 0.044161
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: x-xss-protection: 1; mode=block
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: content-length: 209
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: connection: Close
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: ---- End HTTP Status/Header Data ----
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: ---- HTTP Response Body ----
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: <html><body>You are being <a href="https://s3.amazonaws.com/community-files.opscode.com/cookbook_versions/tarballs/28864/original/fail2ban20190508-41006-1kydj0z.tar.gz?1557352785">redirected</a>.</body></html>
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: ---- End HTTP Response Body -----
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_stream_complete
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: No content-length information collected for the streamed download, cannot identify streamed download.
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: Chef::HTTP calling Chef::HTTP::CookieManager#handle_stream_complete
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: Chef::HTTP calling Chef::HTTP::Decompressor#handle_stream_complete
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: Following redirect 1/10
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: Initiating GET to https://s3.amazonaws.com/community-files.opscode.com/cookbook_versions/tarballs/28864/original/fail2ban20190508-41006-1kydj0z.tar.gz?1557352785
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: ---- HTTP Request Header Data: ----
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: ---- End HTTP Request Header Data ----
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: ---- HTTP Status and Header Data: ----
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: HTTP 1.1 200 OK
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: x-amz-id-2: a2iOnve5iX8paa/pT2s+A21Bm8IvSqbjzYeP8mc2I2kNGSwgj9Wen8IyQlpiQ955grAF7xZKaQE=
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: x-amz-request-id: E74D7A5F59B4B3B0
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: date: Wed, 03 Jul 2019 08:02:26 GMT
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: last-modified: Wed, 08 May 2019 21:59:47 GMT
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: etag: "5c15105980cf9bfb2467ed00b789be0a"
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: accept-ranges: bytes
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: content-type: application/gzip
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: content-length: 9020
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: server: AmazonS3
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: connection: close
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: ---- End HTTP Status/Header Data ----
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: Streaming download from https://supermarket.chef.io/api/v1/cookbooks/fail2ban/versions/6.0.0/download to tempfile /tmp/chef-rest20190703-1-cbdl7x
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: content_encoding = '' initializing noop stream deflator.
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: Chef::HTTP::StreamHandler calling Chef::HTTP::ValidateContentLength::ContentLengthCounter#handle_chunk
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: Chef::HTTP::StreamHandler calling Chef::HTTP::Decompressor::NoopInflater#handle_chunk
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: Chef::HTTP::StreamHandler calling Chef::HTTP::ValidateContentLength::ContentLengthCounter#handle_chunk
worker_1 | [2019-07-03T08:02:25+00:00] TRACE: Chef::HTTP::StreamHandler calling Chef::HTTP::Decompressor::NoopInflater#handle_chunk
So it's like while deflating the fail2ban archive from Amazon S3, it fails due to some encoding stuff.因此,就像从 Amazon S3 压缩 fail2ban 存档时一样,由于某些编码内容而失败。
From that point:从这一点来说:
chef update
command from the terminal, it works fine, no errorschef update
命令时,它工作正常,没有错误 So to me it's like something is missing in my code in order to get this part working, but I can't figure out what could it be.所以对我来说,为了让这部分工作,我的代码中缺少一些东西,但我无法弄清楚它可能是什么。
How could I solve this issue?我怎么能解决这个问题?
PS: The app is running in a Docker container with a Linux Alpine. PS:该应用程序在带有 Linux Alpine 的 Docker 容器中运行。
Chef library is using a Tempfile
in order to write the downloaded data somewhere, so I aliased its write
method in order to call force_encoding('UTF-8')
on the passed argument before to give it back to the original write
method. Chef 库正在使用
Tempfile
来将下载的数据写入某处,因此我将其write
方法设为别名,以便在传递的参数上调用force_encoding('UTF-8')
之前将其返回给原始write
方法。
Here is my override (in a config/initializers/tempfile_override.rb
:这是我的覆盖(在
config/initializers/tempfile_override.rb
:
# frozen_string_literal: true
require 'tempfile'
#
# Overrides the Tempfile write method in order to always use `force_encode` with
# UTF-8 in order to solve the issue where `chef update` fails to write
# downloaded cookbooks data and write it to the disk.
# See https://stackoverflow.com/questions/56866746/encodingundefinedconversionerror-when-calling-chef-update-command-from-a-rails
#
class Tempfile
alias old_write write
def write(data)
old_write(data.force_encoding('UTF-8'))
end
end
I'm kind of forced to do that as I need to change the Chef::UI
object in order to grab the Chef logs in my app.我有点被迫这样做,因为我需要更改
Chef::UI
对象才能在我的应用程序中获取 Chef 日志。
That's not clean, but I don't have a better way to do it without having to patch chef itself.这不干净,但我没有更好的方法来做到这一点,而不必修补厨师本身。
Until something better has been found, this answer will be the accepted one.在找到更好的东西之前,这个答案将是公认的。
If you know a better way to do it, please help me here :)如果您知道更好的方法,请在这里帮助我:)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.