jinja2 '{% set ... %}' processing in ansible playbook

Question

I don't understand how to use jinja2 templating commands in an ansible playbook.

My understanding is that the playbook should be processed as a jinja2 template before execution, but apparently not before the file is parsed as yaml, because using a jinja2 command at the top of the file yields syntax errors, eg:

{% set test_var = "test_value" %}
- hosts: all
  remote_user: "my_user"
  tasks:
    - debug: var=test_var
    {% set another_var = "another_value" %}
    - debug: var=another_var

$ ansible-playbook -vv -K ansible/playbooks/test.yml
ERROR! Syntax Error while loading YAML.
  found character '%' that cannot start any token
...

If I comment the jinja2 commands to avoid this parse error, the first command at the top is processed, but not others in the middle of the playbook:

# {% set test_var = "test_value" %}
- hosts: all
  remote_user: "my_user"
  tasks:
    - debug: var=test_var # this works
    # {% set another_var = "another_value" %}
    - debug: var=another_var
    # ok: [localhost] => {
    #   "another_var": "VARIABLE IS NOT DEFINED!: 'another_var' is undefined"
    # }

I don't understand how ansible is processing playbook templating. Shouldn't there be a first pass that processes only jinja2 syntax, then outputs the yaml with jinja2 syntax removed?

$ ansible-playbook --version
ansible-playbook 2.9.1
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/me/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/me/.local/lib/python3.7/site-packages/ansible
  executable location = /home/me/.local/bin/ansible-playbook
  python version = 3.7.3 (default, Apr  3 2019, 19:16:38) [GCC 8.0.1 20180414 (experimental) [trunk revision 259383]]

Thanks!

Answer 1

What is processed by jinja2 at ansible runtime is the content of the yaml vars, not the yaml file itself. This is done in several passes, dig in the ansible code if you want to understand what is under the hood exactly.

Just to put you on track, this is the ansible way of doing what you tried in your question.

- hosts: all
  remote_user: "my_user"

  # These are playbook vars set at playbook level
  # Vars can also be in inventories, roles....
  # see https://docs.ansible.com/ansible/latest/user_guide/playbooks_variables.html
  vars:
    test_var: test_value

  tasks:

    - name: debug my test_var
      debug:
        var: test_var

    - name: if you need to set var at run time you can use set_fact
      set_fact:
        # Jinja2 templating will be honored here
        another_var: "another_{{ test_var }}"

    - name: debug var we just valued
      debug:
        var: another_var