如何在气隙环境中获取 GitLab CI 跑步者上的 Terraform 提供者

Question

I am running a Gitlab CI runner on an Azure Red Hat Linux 7.9 VM which is air-gapped and cannot communicate with the outside world due to our network restrictions. In the.gitlab-ci.yml file of my GitLab pipeline, I run some Terraform commands including Terraform init which subsequently attempts to pull down the Terraform provider plugins from the default Hashicorp location.

Obviously, with the prevailing network restrictions, the aforementioned Terraform command fails and subsequently the entire pipeline job fails. From what I've now established and bearing in mind our particular network setup and circumstances, I probably have two options:

1. Configure my GitLab pipeline with a Proxy to allow access to sites like registry.terraform.io. I have been provided a proxy address by the way, but I'm not entirely sure how to get this configured in GitLab. Will therefore appreciate some assistance with this.

2. It also appears I will be able to force my Terraform configuration to reference the required provider plugins locally or on an internal share. Again, any advice on how to set this up would be greatly appreciated, having spent hours poring over endless online material on both options without any success.

Answer 1

Configure my GitLab pipeline with a Proxy

If you have a forward proxy available in your environment and want to have your CI jobs use that proxy, it's usually simply a matter of setting your HTTP_PROXY and HTTPS_PROXY variables. You can do this in your runner environment configuration or CI configuration. For example in a .gitlab-ci.yml you could add:

variables:
  HTTP_PROXY: "http://proxy.mydomain.corp"
  HTTPS_PROXY: "http://proxy.mydomain.corp"

Refer to the official docs Running GitLab Runner Behind a Proxy for more information.

---

It also appears I will be able to force my Terraform configuration to reference the required provider plugins locally or on an internal share

You can use the filesystem mirror option to configure a terraform providers mirror that lives on your filesystem (requires terraform v0.13+).

By default, the directory .terraform.d/plugins can be used without any additional configuration (see implied mirrors ). When you have the provider files in your filesystem mirror, you won't need to download them from the internet.

Let's say for example you're using the graylog provider like this:

provider "graylog" {
  version          = "1.0.4"
  web_endpoint_uri = "https://graylog.example.com/api"
  api_version      = "v3"
  auth_name        = "admin"
  auth_password    = "password"
}

All you need are the requisite files -- namely, the zip files of the providers you need (eg, terraform-provider-graylog_1.0.4_linux_amd64.zip ), then in your job copy them into your .terraform.d plugins directory (eg, .terraform.d/plugins/... ) in an expected layout (packed or unpacked).

You can make these files available by placing them on your runner and (for docker runners) mounting the files into the job using the volumes configuration .

For example, say you have the providers on your runner host, you can mount them into your jobs with this configuration in the config.toml file (only needed for docker-based runners!)

[runners.docker]
  # ...
  volumes = ["/path/to/plugins/from/host:/plugins/path/in/container:rw"]

Then in your jobs, you can copy the provider files into the .terraform directory.

my_job:
  image: 
    name: hashicorp/terraform
    entrypoint: [""]
  before_script:
    - cp /plugins/path/in/container/ .terraform.d/plugins/ 
    - terraform init  # will get plugins from your local mirror