Ansible AWX issue - JMESPATH

Question

I am facing an issue with AWX Tower instance.

I have a project and job template on it and the job template is connecting to Ansible control node running on (Red Hat Enterprise Linux Server release 7.9 (Maipo)), where couple of playbooks are executed.

In one of the playbooks I am using the following task:

- name: Filter with json_query the list of current assigned policies of the LDAP user as string separated by comma
  set_fact: ldap_policy_response: "{{ ldap_user_current_policies.json | community.general.json_query(jmesquery) | join(',') }}" 
  vars: 
    jmesquery: 'data.policies[*]' # ldap_policy_response

When I run the playbook from the Ansible Control Node directly, everything works fine and jmesquery is executed.

I have a jmespath installed on the server, but when I try to run the playbooks from AWX Tower the task mentioned above fails with thrown error

"You need to install \"jmespath\" prior to running json_query filter"

Any idea what might be the reason for this behave, and what might be the issue?

jmespath version

Installed Packages
python2-jmespath.noarch   0.9.4-2.el7   u/rhel-7-server-epel7-x86_64

Ansible version details

ansible 2.9.27
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/ansible/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /bin/ansible
  python version = 2.7.5 (default, May 27 2022, 11:27:32) [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)]

Answer 1

In a similar environment, RHEL 7.9.5, Ansible 2.9.27, Python 2.7.5, I've ran a small playbook which contains a json_query for testing and found it executing without an error.

I have a "jmespath" installed on the server, but when I try to run the playbooks from AWX Tower the task mentioned above fails with thrown error - You need to install \"jmespath\" prior to running json_query filter

The reason why my playbook is running and you are receiving an error is probably the environment.

Whereby neither under my user, nor under the user Ansible Tower is running ( awx ), the Python package is installed

user:~$ pip list | grep jmespath; echo $?
1
user:~$ sudo su - awx
-bash-4.2$ pip list | grep jmespath; echo $?
1

it is installed read-only for all users (under root).

user:~$ sudo pip list | grep jmespath
jmespath (0.9.4)

I haven't installed the required library via native OS package management

user:~$ sudo yum provides jmespath
...
python2-jmespath-0.9.0-3.el7.noarch : JSON Matching Expressions
Repo        : EXTRAS
Matched from:
Filename    : /usr/lib/python2.7/site-packages/jmespath
...
python2-jmespath-0.9.4-2.el7.noarch : JSON Matching Expressions
Repo        : EPEL-7
Matched from:
Filename    : /usr/lib/python2.7/site-packages/jmespath
...

user:~$ sudo yum list installed jmespath; echo $?
Error: No matching Packages to list
1

but via pip

user:~$ sudo find / -name jmespath
/var/lib/awx/venv/ansible/lib/python2.7/site-packages/jmespath
/usr/lib/python2.7/site-packages/jmespath

Debugging the runtime environment with a task like

- name: Show environment
  debug:
    msg: "{{ ansible_env }}"

providing an output like

...
            "PYTHONPATH": "/var/lib/awx/venv/ansible/lib/python2.7/site-packages", 
...
            "USER": "awx", 
            "VIRTUAL_ENV": "/var/lib/awx/venv/ansible", 
...

might help to narrow down the issue.