Ansible playbook stops working in new Ubuntu/Ansible/Python environment

Question

I have a working Ansible setup in one environment, and I cannot get it to run properly in my new environment.

I get the following error when I run in the new environment and am unsure where to narrow down the issue (Python mismatch? Ansible version mismatch?).

Any suggestion on how to narrow this down would be greatly appreciated.

TASK [Provision FGT1 Device] ************************************************************************************************************
task path: /home/fortinet/Downloads/ansible/playbooks/complete_provisioning/ansible_provision_playbook.yml:6
Using module file /home/fortinet/Downloads/ansible/modules/fmgr_provisioning.py
<192.168.0.120> ESTABLISH LOCAL CONNECTION FOR USER: root
<192.168.0.120> EXEC /bin/sh -c 'echo ~ && sleep 0'
<192.168.0.120> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1592597614.94-254333872560613 `" && echo a   nsible-tmp-1592597614.94-254333872560613="` echo /root/.ansible/tmp/ansible-tmp-1592597614.94-254333872560613 `" ) && sleep 0'
<192.168.0.120> PUT /root/.ansible/tmp/ansible-local-25254o2_iRu/tmpqOs7iz TO /root/.ansible/tmp/ansible-tmp-1592597614.94-25433387256061   3/fmgr_provisioning.py
<192.168.0.120> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1592597614.94-254333872560613/ /root/.ansible/tmp/ansible-tmp-1   592597614.94-254333872560613/fmgr_provisioning.py && sleep 0'
<192.168.0.120> EXEC /bin/sh -c '/usr/bin/python /root/.ansible/tmp/ansible-tmp-1592597614.94-254333872560613/fmgr_provisioning.py && sle   ep 0'
<192.168.0.120> EXEC /bin/sh -c 'rm -f -r /root/.ansible/tmp/ansible-tmp-1592597614.94-254333872560613/ > /dev/null 2>&1 && sleep 0'

The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_422O5V/ansible_module_fmgr_provisioning.py", line 624, in <module>
    main()
  File "/tmp/ansible_422O5V/ansible_module_fmgr_provisioning.py", line 512, in main
    response = fmg.login()
  File "/tmp/ansible_422O5V/ansible_modlib.zip/ansible/module_utils/network/fortimanager/fortimanager.py", line 59, in login
  File "/usr/local/lib/python2.7/dist-packages/pyFMG/fortimgr.py", line 476, in login
    self._lock_ctx.check_mode()
  File "/usr/local/lib/python2.7/dist-packages/pyFMG/fortimgr.py", line 99, in check_mode
    code, resp_obj = self._fmg.get(url, fields=["workspace-mode", "adom-status"])
  File "/usr/local/lib/python2.7/dist-packages/pyFMG/fortimgr.py", line 517, in get
    return self._post_request("get", self.common_datagram_params("get", url, *args, **kwargs))
  File "/usr/local/lib/python2.7/dist-packages/pyFMG/fortimgr.py", line 371, in _post_request
    raise FMGValidSessionException(method, params)
pyFMG.fortimgr.FMGValidSessionException: A call using the get method was requested to /cli/global/system/global on a FortiManager instanc   e that had no valid session or was not connected. Paramaters were:
[{'url': '/cli/global/system/global', 'fields': ['workspace-mode', 'adom-status']}]

fatal: [192.168.0.120]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_422O5V/ansible_module_fmgr_provisioning.py\", line 624, i   n <module>\n    main()\n  File \"/tmp/ansible_422O5V/ansible_module_fmgr_provisioning.py\", line 512, in main\n    response = fmg.login()   \n  File \"/tmp/ansible_422O5V/ansible_modlib.zip/ansible/module_utils/network/fortimanager/fortimanager.py\", line 59, in login\n  File    \"/usr/local/lib/python2.7/dist-packages/pyFMG/fortimgr.py\", line 476, in login\n    self._lock_ctx.check_mode()\n  File \"/usr/local/li   b/python2.7/dist-packages/pyFMG/fortimgr.py\", line 99, in check_mode\n    code, resp_obj = self._fmg.get(url, fields=[\"workspace-mode\"   , \"adom-status\"])\n  File \"/usr/local/lib/python2.7/dist-packages/pyFMG/fortimgr.py\", line 517, in get\n    return self._post_request   (\"get\", self.common_datagram_params(\"get\", url, *args, **kwargs))\n  File \"/usr/local/lib/python2.7/dist-packages/pyFMG/fortimgr.py\   ", line 371, in _post_request\n    raise FMGValidSessionException(method, params)\npyFMG.fortimgr.FMGValidSessionException: A call using    the get method was requested to /cli/global/system/global on a FortiManager instance that had no valid session or was not connected. Para   maters were:\n[{'url': '/cli/global/system/global', 'fields': ['workspace-mode', 'adom-status']}]\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE",
    "rc": 1
}
        to retry, use: --limit @/home/fortinet/Downloads/ansible/playbooks/complete_provisioning/ansible_provision_playbook.retry

PLAY RECAP ******************************************************************************************************************************
192.168.0.120              : ok=0    changed=0    unreachable=0    failed=1

Here is an "ansible --version" output of the working setup

fortinet@Ansible-Host:~$ ansible --version
ansible 2.7.11
  config file = /home/fortinet/.ansible.cfg
  configured module search path = [u'/home/fortinet/ansible/modules']
  ansible python module location = /usr/local/lib/python2.7/dist-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.12 (default, Nov 12 2018, 14:36:49) [GCC 5.4.0 20160609]
fortinet@Ansible-Host:~$

And here is an "ansible --version" output of the non-working setup

sysadmin@ubuntu18:/home/fortinet/Downloads/ansible/playbooks/complete_provisioning$ ansible --version
ansible 2.5.1
  config file = /home/fortinet/Downloads/ansible/playbooks/complete_provisioning/ansible.cfg
  configured module search path = [u'/home/fortinet/Downloads/ansible/modules']
  ansible python module location = /usr/lib/python2.7/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.15rc1 (default, Nov 12 2018, 14:31:15) [GCC 7.3.0]
sysadmin@ubuntu18:/home/fortinet/Downloads/ansible/playbooks/complete_provisioning$

I also get an error when I try to run the playbook using an older version of Python I just installed (2.7.12)

ansible-playbook ansible_provision_playbook.yml -vvv -e 'ansible_python_interpreter=/usr/local/bin/python'
ansible-playbook 2.5.1
  config file = /home/fortinet/Downloads/ansible/playbooks/complete_provisioning/ansible.cfg
  configured module search path = [u'/home/fortinet/Downloads/ansible/modules']
  ansible python module location = /usr/lib/python2.7/dist-packages/ansible
  executable location = /usr/bin/ansible-playbook
  python version = 2.7.15rc1 (default, Nov 12 2018, 14:31:15) [GCC 7.3.0]
Using /home/fortinet/Downloads/ansible/playbooks/complete_provisioning/ansible.cfg as config file
Parsed /home/fortinet/Downloads/ansible/playbooks/complete_provisioning/hosts inventory source with ini plugin

PLAYBOOK: ansible_provision_playbook.yml ***********************************************************************************************************************************************
1 plays in ansible_provision_playbook.yml

PLAY [Processing Devices for Automated Provisioning] ***********************************************************************************************************************************
META: ran handlers

TASK [Provision FGT1 Device] ***********************************************************************************************************************************************************
task path: /home/fortinet/Downloads/ansible/playbooks/complete_provisioning/ansible_provision_playbook.yml:6
Using module file /home/fortinet/Downloads/ansible/modules/fmgr_provisioning.py
<192.168.0.120> ESTABLISH LOCAL CONNECTION FOR USER: sysadmin
<192.168.0.120> EXEC /bin/sh -c 'echo ~ && sleep 0'
<192.168.0.120> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/sysadmin/.ansible/tmp/ansible-tmp-1592623661.17-74974994506330 `" && echo ansible-tmp-1592623661.17-74974994506330="` echo /home/sysadmin/.ansible/tmp/ansible-tmp-1592623661.17-74974994506330 `" ) && sleep 0'
<192.168.0.120> PUT /home/sysadmin/.ansible/tmp/ansible-local-6279tjyzd_/tmp0yJ5I1 TO /home/sysadmin/.ansible/tmp/ansible-tmp-1592623661.17-74974994506330/fmgr_provisioning.py
<192.168.0.120> EXEC /bin/sh -c 'chmod u+x /home/sysadmin/.ansible/tmp/ansible-tmp-1592623661.17-74974994506330/ /home/sysadmin/.ansible/tmp/ansible-tmp-1592623661.17-74974994506330/fmgr_provisioning.py && sleep 0'
<192.168.0.120> EXEC /bin/sh -c '/usr/local/bin/python /home/sysadmin/.ansible/tmp/ansible-tmp-1592623661.17-74974994506330/fmgr_provisioning.py && sleep 0'
<192.168.0.120> EXEC /bin/sh -c 'rm -f -r /home/sysadmin/.ansible/tmp/ansible-tmp-1592623661.17-74974994506330/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/home/sysadmin/.ansible/tmp/ansible-tmp-1592623661.17-74974994506330/fmgr_provisioning.py", line 123, in <module>
    f.write(z.read('ansible_module_fmgr_provisioning.py'))
  File "/usr/local/lib/python2.7/zipfile.py", line 935, in read
    return self.open(name, "r", pwd).read()
  File "/usr/local/lib/python2.7/zipfile.py", line 1010, in open
    close_fileobj=should_close)
  File "/usr/local/lib/python2.7/zipfile.py", line 526, in __init__
    self._decompressor = zlib.decompressobj(-15)
AttributeError: 'NoneType' object has no attribute 'decompressobj'

fatal: [192.168.0.120]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/home/sysadmin/.ansible/tmp/ansible-tmp-1592623661.17-74974994506330/fmgr_provisioning.py\", line 123, in <module>\n    f.write(z.read('ansible_module_fmgr_provisioning.py'))\n  File \"/usr/local/lib/python2.7/zipfile.py\", line 935, in read\n    return self.open(name, \"r\", pwd).read()\n  File \"/usr/local/lib/python2.7/zipfile.py\", line 1010, in open\n    close_fileobj=should_close)\n  File \"/usr/local/lib/python2.7/zipfile.py\", line 526, in __init__\n    self._decompressor = zlib.decompressobj(-15)\nAttributeError: 'NoneType' object has no attribute 'decompressobj'\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE",
    "rc": 1
}
 [WARNING]: Could not create retry file '/home/fortinet/Downloads/ansible/playbooks/complete_provisioning/ansible_provision_playbook.retry'.         [Errno 13] Permission denied:
u'/home/fortinet/Downloads/ansible/playbooks/complete_provisioning/ansible_provision_playbook.retry'


PLAY RECAP *****************************************************************************************************************************************************************************
192.168.0.120              : ok=0    changed=0    unreachable=0    failed=1

Answer 1

This particular problem turned out to be an issue of Ansible mismatch. The original setup was Ansible 2.7, and since I couldn't get this working with Ansible 2.9, I used a system that was Ansible 2.5.

Upgrading this from 2.5 to 2.7 "fixed" this particular error, but left me with several other problems.

Most of these issues seem to stem from the way that playbooks are run now via httpapi versus localhost, and I will have to create new playbooks for my previous FortiGate/FortiManager tasks.

https://docs.ansible.com/ansible/latest/plugins/connection/httpapi.html

(This change apparently happened in Ansible 2.8. I'm new to Ansible, so am still getting up to speed on what these changes mean for playbooks.)

My permanent fix for this to was a Dockerfile that ensure that my scripting environment stays constant on each platform.