I am currently setting up a number of Ansible roles to setup a Kubernetes cluster. So far I have a role to provision idempotent EC2s (1x Master / 2x Worker) and subsequent roles to setup these master/worker nodes with Docker/Kubernetes dependencies. I am using AWS ec2.ini/.py dynamic-inventory
to discover IPs of the instances provisioned by my create_ec2
role.
I have encountered an issue when trying to join my workers to the cluster with the join command I am retrieving from the master node. I have 2 seperate roles for the master & worker provisioning. In the tasks for the master, I get the join command with:
kubeadm token create --print-join-command
and then register a variable which I then use to set a host fact:
set_fact:
join_command: "{{ join_command_stdout.stdout_lines[0] }}"
The issue I am having is when I try to access this fact on my worker nodes when running my worker role. I am trying to access the fact with:
"{{ hostvars['tag_Type_master'].join_command }} --ignore-preflight-errors all >> node_joined.txt"
However it is failing as the host I am providing for the hostvars is apparently undefined..
For reference, I have this value held in my dynamic-inventory (IP omitted):
"tag_Type_master": [
"1.2.3.4"
The error I am receiving is:
"{"msg": "The task includes an option with an undefined variable. The error was: \"hostvars['tag_Type_master']\" is undefined"
I am struggling to figure out how I access the host facts of an EC2 instance defined in my dynamic-inventory.
I have tried supplementing the EC2 IP directly into the hostvars ( hostvars['1.2.3.4'].join_command
), however the task just hangs and does nothing.
I have also tried putting in a Magic variable ( hostvars['inventory_hostname].join_command
) to no avail.
It seems that people have had success with accessing host facts from hosts defined in a static inventory file, however due to the dynamic nature of the EC2 servers the cluster will be created on I am unable to use this approach.
name: Setup K8s master node
hosts: tag_Name_kube_master
gather_facts: true
roles:
- setup_kube_master
name: Setup K8s worker nodes
hosts: tag_Name_kube_worker
gather_facts: true
roles:
- setup_kube_worker
name: Get join command for workers
shell: kubeadm token create --print-join-command
register: join_command_stdout
name: Persist variable for workers
set_fact:
join_command: "{{ join_command_stdout.stdout_lines[0] }}"
name: join cluster
shell: "{{ hostvars['tag_Type_master'].join_command }} --ignore-preflight-errors all >> node_joined.txt"
args:
chdir: $HOME
creates: node_joined.txt
So the way you would troubleshoot this for yourself is to use the debug:
task to show the entire fact cache and find the relationship for yourself:
- name: show the state of affairs
debug: var=hostvars verbosity=0
However, having said that, I'm pretty sure that tag_Type_master
is defined as a group and thus will not show up in hostvars
since -- as its name implies -- it is vars
for hosts not vars
for groups
You have to do one level of indirection to obtain a host that is a member of that group:
- hosts: tag_Name_kube_worker
tasks:
- name: promote the "join_command" fact from the masters group
set_fact:
join_command: '{{ some_master.join_command }}'
vars:
some_master: '{{ hostvars[groups["tag_Type_master"][0]] }}'
I took some liberties with that some_master
definition for the sake of brevity -- in production code you would want to actually check that that group exists and its contents are not empty, etc, etc, but I'm about 80% sure it will work even as written
You would want that to appear in run.yml
in between the hosts: tag_Type_master
and hosts: tag_Type_worker
to bridge the fact-gap between the two groups and make it appear as if the workers had that join_command
fact the whole time
Separately, while this isn't what you asked, if you were to tag those instances with "kubernetes.io/role/master": ""
and/or "kubernetes.io/role": "master"
you would benefit by already having the tags that the cloud-provider
is expecting . I have no idea what that would look like in ec2.py
, but I'm sure it would be cheap to find out using ansible-inventory -i ec2.py --list
I tag the workers with the corresponding kubernetes.io/role: worker
even though I'm pretty sure the AWS cloud-provider doesn't care about it, choosing instead to just use the metadata.labels
on the existing Nodes for doing ELB registration et al.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.