在 VMware Ubuntu 22.04 客戶機上通過 Packer 和 Terraform 運行 cloud-init 兩次

Question

我想從 ESXi 中的同一個 Ubuntu 模板創建具有不同 CPU、RAM 和網絡配置的虛擬機。

當$ packer build -var-file=packer/variables.pkr.hcl -var-file=packer/secret.pkrvars.hcl packer/template.pkr.hcl運行時，它會讀取以下packer/template.pkr.hcl ：

variable "vm_name" {
  type = string
  default = "Ubuntu_Server_22.04_LTS"
}

variable "esxi_password" {
  type =  string
  default = "password"
  sensitive = true
}

variable "vm_password" {
  type =  string
  default = "password"
  sensitive = true
}

source "vmware-iso" "ubuntu-2204" {
  vm_name = "${var.vm_name}"
  guest_os_type = "ubuntu-64"

  iso_checksum = "sha256:84aeaf7823c8c61baa0ae862d0a06b03409394800000b3235854a6b38eb4856f"
  iso_url = "https://REDACTED/ubuntu-22.04-live-server-amd64.iso"

  http_directory = "/home/REDACTED/packer/http"

  shutdown_command = "sudo shutdown -P now"

  remote_type = "esx5"

  remote_datastore = "REDACTED"
  remote_host = "REDACTED"
  remote_username = "REDACTED"
  remote_password = "${var.esxi_password}"
  remote_private_key_file = ""

  cpus = 8
  memory = 16384
  disk_size = 16384

  network_adapter_type = "vmxnet3"
  network_name = "REDACTED"

  headless = false
  vnc_over_websocket = true
  insecure_connection = true

  tools_upload_flavor = "linux"

  skip_export = true
  keep_registered = true

  ssh_username = "REDACTED"
  ssh_password = "${var.vm_password}"
  ssh_timeout = "15m"
  ssh_handshake_attempts = "100"

  boot_wait = "3s"
  boot_command = [
    "<esc><esc><esc><esc>e<wait>",
    "<del><del><del><del><del><del><del><del><del><del>",
    "<del><del><del><del><del><del><del><del><del><del>",
    "<del><del><del><del><del><del><del><del><del><del>",
    "<del><del><del><del><del><del><del><del><del><del>",
    "<del><del><del><del><del><del><del><del><del><del>",
    "<del><del><del><del><del><del><del><del><del><del>",
    "<del><del><del><del><del><del><del><del><del><del>",
    "<del><del><del><del><del><del><del><del><del><del>",
    "<del><del><del><del><del><del><del><del><del><del>",
    "<del><del><del><del><del><del><del><del><del><del>",
    "<del><del><del><del><del><del><del><del><del><del>",
    "<del><del><del>",
    "linux /casper/vmlinuz --- autoinstall ds=\"nocloud-net;seedfrom=http://[{{.HTTPIP}}]:{{.HTTPPort}}/\"<enter><wait>",
    "initrd /casper/initrd<enter><wait>",
    "boot<enter>",
    "<enter><f10><wait>"
  ]
}

build {
  sources = ["sources.vmware-iso.ubuntu-2204"]
  provisioner "shell" {
    inline = [
      "ls /"
    ]
  }
}

packer/http/user-data包含以下內容：

#cloud-config
autoinstall:
  version: 1
  early-commands:
    # Stop SSH to prevent Packer from connecting too early
    - systemctl stop ssh
  apt:
    preserve_sources_list: false
    primary:
    - arches: [amd64, i386]
      uri: https://REDACTED
    - arches: [default]
      uri: http://ports.ubuntu.com/ubuntu-ports
  locale: en_US
  keyboard:
    layout: en
    variant: us
  network:
    version: 2
    renderer: networkd
    ethernets:
      ens160:
        dhcp4: true
        dhcp-identifier: mac
        dhcp6: true
  storage:
    layout:
      name: direct
    config:
      - type: disk
        id: disk0
        match:
          size: largest
      - type: partition
        id: boot-partition
        device: disk0
        size: 500M
      - type: partition
        id: root-partition
        device: disk0
        size: -1
  ssh:
    install-server: true
    allow-pw: true
    authorized-keys:
      - ssh-ed25519 REDACTED
  identity:
    hostname: ubuntu
    username: REDACTED
    password: REDACTED
  packages:
    - open-vm-tools
    - python3
  late-commands:
    - echo 'REDACTED ALL=(ALL) NOPASSWD:ALL' > /target/etc/sudoers.d/REDACTED
    - curtin in-target --target=/target -- chmod 440 /etc/sudoers.d/REDACTED
    - curtin in-target --target=/target -- apt-get update
    - curtin in-target --target=/target -- apt-get upgrade --yes
    - curtin in-target --target=/target -- sudo cloud-init clean

這將創建一個 Ubuntu 22.04 服務器模板，然后我可以使用 Terraform 從中配置虛擬機。

# cat /var/log/installer/autoinstall-user-data在 VM 上運行表明 Packer 已成功提供用戶數據。 它已被執行，我通過 SSH 登錄的能力證明了這一點。

當$ terraform apply -var-file=secret.tfvars在我的 terraform 目錄中運行時，它會讀取以下main.tf和https://github.com/josenk/terraform-provider-esxi提供的提供provider "esxi" ：

variable "vm_name" {
  description = "The name of the virtual machine"
  default     = "ubuntu-terraformed"
  type        = string
}

variable "esxi_password" {
  description = "The password for the ESXi root user"
  type        = string
}

provider "esxi" {
  esxi_hostname = "REDACTED"
  esxi_username = "REDACTED"
  esxi_password = "${var.esxi_password}"
}

data "template_file" "Test" {
  template = file("userdata.tpl")
  vars = {
    HOSTNAME = "${var.vm_name}"
  }
}

resource "esxi_guest" "Test" {
  guest_name = "${var.vm_name}"
  disk_store = "REDACTED"

  clone_from_vm = "Ubuntu_Server_22.04_LTS"

  network_interfaces {
    virtual_network = "REDACTED"
    nic_type = "vmxnet3"
  }

  network_interfaces {
    virtual_network = "REDACTED"
    nic_type = "vmxnet3"
  }

  network_interfaces {
    virtual_network = "REDACTED"
    nic_type = "vmxnet3"
  }

  network_interfaces {
    virtual_network = "REDACTED"
    nic_type = "vmxnet3"
  }

  network_interfaces {
    virtual_network = "REDACTED"
    nic_type = "vmxnet3"
  }

  network_interfaces {
    virtual_network = "REDACTED"
    nic_type = "vmxnet3"
  }

  network_interfaces {
    virtual_network = "REDACTED"
    nic_type = "vmxnet3"
  }

  guestinfo = {
    "userdata.encoding" = "gzip+base64"
    "userdata"          = base64gzip(data.template_file.Test.rendered)
  }
}

userdata.tpl包含以下內容：

#cloud-config

hostname: ${HOSTNAME}
network:
  version: 2
  renderer: networkd
  ethernets:
    ens160:
      dhcp4: true
      dhcp-identifier: mac
      dhcp6: true
    ens161:
      dhcp4: true
      dhcp-identifier: mac
      dhcp6: true
    ens192:
      dhcp4: true
      dhcp-identifier: mac
      dhcp6: true
    ens193:
      dhcp4: true
      dhcp-identifier: mac
      dhcp6: true
    ens224:
      dhcp4: true
      dhcp-identifier: mac
      dhcp6: true
    ens225:
      dhcp4: true
      dhcp-identifier: mac
      dhcp6: true
    ens256:
      dhcp4: true
      dhcp-identifier: mac
      dhcp6: true
package_upgrade: true
#ntp:
#  enabled: true
#  servers:
#    - REDACTED
#timezone: REDACTED
#late-commands:
#  - curtin in-target --target=/target -- sudo sed -i 's/#NTP=/NTP=REDACTED/g' /etc/systemd/timesyncd.conf
#  - curtin in-target --target=/target -- sudo timedatectl set-ntp true
#  - curtin in-target --target=/target -- sudo timedatectl set-timezone REDACTED
#  - curtin in-target --target=/target -- sudo systemctl restart systemd-timesyncd.service

這將基於帶有正確來賓參數的打包程序模板創建一個 VM。 VMware 來賓配置包含我已驗證的 userdata 屬性與 base64 解碼和解壓縮來賓參數后提供的用戶數據相匹配。

我遇到的問題是 VM 似乎不包含或執行“第二個”Terraform cloud-init 用戶數據。

/var/lib/cloud/instance/user-data.txt 沒有顯示第二個配置：

#cloud-config
growpart:
  mode: 'off'
locale: en_US.UTF-8
preserve_hostname: true
resize_rootfs: false
ssh_pwauth: true
users:
- gecos: me
  groups: !!set
    adm: null
    cdrom: null
    dip: null
    lxd: null
    plugdev: null
    sudo: null
  lock_passwd: false
  name: me
  passwd: REDACTED
  shell: /bin/bash
  ssh_authorized_keys:
  - ssh-ed25519 REDACTED
    me

/var/log/cloud-init-output.log 顯示：

schema.py[WARNING]: Invalid cloud-config provided:
users.0: {'gecos': 'me', 'groups': {'sudo', 'lxd', 'cdrom', 'adm', 'plugdev', 'dip'}, 'lock_passwd': False, 'name': 'me', 'passwd': 'REDACTED', 'shell': '/bin/bash', 'ssh_authorized_keys': ['ssh-ed25519 REDACTED me']} is not valid under any of the given schemas

當我的工作流程是 Packer 生成模板時，嘗試應用用戶數據兩次，而 Terraform 使用該模板應用新 VM，對嗎？ 如果是這樣，我最好在哪里找出為什么 Ubuntu 不包含並執行第二次迭代？

順便說一句，如果我有任何其他建議我應該考慮，請隨時發表評論。 我想我應該能夠通過即將實施的 Ansible 計划來實現這一點，但我仍然需要設置諸如主機名之類的選項。

Answer 1

我能夠通過將 Packer 構建更改為以下內容來解決此問題：

build {
  sources = ["sources.vmware-iso.ubuntu-2204"]
  provisioner "shell" {
    inline = [
      "sudo rm -f /etc/cloud/cloud.cfg.d/99-installer.cfg",
      "sudo rm -f /etc/cloud/cloud.cfg.d/subiquity-disable-cloudinit-networking.cfg",
      "echo 'disable_vmware_customization: false' | sudo tee -a /etc/cloud/cloud.cfg",
      "sudo sed -i 's|nocloud-net;seedfrom=http://.*/|vmware|' /etc/default/grub",
      "sudo update-grub",
      "sudo cloud-init clean"
    ]
  }
}

以下文章中引用了 cloud.cfg 調整，並允許通過 cloud-init 配置操作系統網絡，這是讓這一切正常工作的最后一步：

調整 GRUB 引導命令，並將cloud-init clean附加到 Packer 構建配置步驟可確保在 Packer 構建的客戶機最終關閉之前，VM 將不再嘗試從初始預置啟動，而是將使用下次啟動時指定的 VM guestinfo。

我還將 Terraform 用戶數據拆分為用於實例標識和網絡配置的元數據，將現在未注釋的 NTP 和時區配置留在用戶數據中。

元數據.tpl：

#cloud-config

instance-id: ${HOSTNAME}
local-hostname: ${HOSTNAME}
network:
  version: 2
  ethernets:
    nics:
      match:
        name: ens*
      dhcp-identifier: mac
      dhcp4: yes
      dhcp6: yes

用戶數據.tpl：

#cloud-config

package_upgrade: true
ntp:
  enabled: true
  servers:
    - REDACTED
timezone: REDACTED

順便說一句，配置網絡接口的更簡單方法是使用 match 指令而不是顯式指定接口。

所有項目現在都按預期配置。 可能僅在使用vmware-iso的 ESXi 中才需要此配置，因為我沒有看到其他地方需要這些步驟。

gecos 警告似乎是一個紅鯡魚，我沒有進一步嘗試解決這個問題。

在 VMware Ubuntu 22.04 客戶機上通過 Packer 和 Terraform 運行 cloud-init 兩次

問題描述

1 個解決方案

解決方案1
0 已采納 2022-06-27 07:41:36

在 VMware Ubuntu 22.04 客戶機上通過 Packer 和 Terraform 運行 cloud-init 兩次

問題描述

1 個解決方案

解決方案1 0 已采納 2022-06-27 07:41:36

解決方案1
0 已采納 2022-06-27 07:41:36