Ansible/Rackspace - adding multiple nodes to a load balancer

Question

I'm using Ansible to build multiple nodes and then add them to a load balancer. The issue that arises is that when the second node is being added using the rax_clb_nodes module, I get:

TASK: [build-servers | add nodes to load balancer] ************************** 
failed: [myserver-v0-0-1-ord -> 127.0.0.1] => {"failed": true}
msg: Load Balancer '123456' has a status of 'PENDING_UPDATE' and is considered immutable.
changed: [myserver-v0-0-1-ord -> 127.0.0.1]

my playbook defines wait=yes and wait_timeout=60 in rax_clb_nodes, so I'm unsure as to why this occurs.

Any insight into a fix?

Answer 1

This is happening because the rax_clb_nodes task is being executed in parallel on behalf of each matching host in your inventory, but only one load balancer update can be performed at a time. There are a few approaches to executing it in serial, instead, depending on how your playbooks are organized.

If this task can be easily moved into its own play, use serial: 1 to execute it on behalf of one host at a time instead:

- name: Load balancer membership
  hosts: build-servers
  serial: 1
  tasks:

  - name: Add {{ inventory_hostname }} node to the load balancer
    local_action:
      module: rax_clb_nodes
      address: "{{ rax_addresses.private.0.addr }}"
      port: 443
      state: present
      wait: yes
      wait_timeout: 60
      # ...

Unfortunately, serial can only be specified on entire plays at a time, not individual tasks. If the task can't easily be moved to its own play (for example, if it's within a role or in the midst of an included task list), you can achieve serial execution by a combination of run_once and a with_items loop :

- name: Add nodes to load balancer
  local_action:
    module: rax_clb_nodes
    address: "{{ hostvars[item].rax_addresses.private.0.addr }}"
    port: 443
    state: present
    wait: yes
    wait_timeout: 60
    # ...
  run_once: yes
  with_items: "{{ groups['build-servers'] }}"