Cannot get AWS SAM CLI to work with Python 3.5.x

Question

I am certain that this issue is down to my own ignorance of Python and pip, however, I have scoured the net and tried many things to get this to work, so far, to no avail.

I have a docker image based on the Bitbucket pipelines default image (version 2): atlassian/default-image:2 , which uses Python 2.7.x. And SAM doesn't work with that, I looked at AWS' documentation and this said that the SAM CLI no longer supports Python 2.7.x, so I removed all Python installations from the docker container (debugging the Bitbucket pipeline locally) and attempted to make sure that there was only Python3 and pip3 installed (I even setup symlinks from /usr/bin/python to /usr/bin/python3 and from /usr/bin/pip to /usr/bin/pip3 - although, that seems like such a dumb thing to have to do).

I also tried doing the above with Python 3.6, as the error message (shown below) seemed to show a Python 'f-String' syntax, which according to Python's docs was introduced in 3.6.

Nothing I do seems to work and the more I look at this, the more I get tied up in an seemingly never ending web of Python versions and linux packages!

The error (and python versions) can be seen here:

root@eb3dc6ff9997:/localDebugRepo/infrastructure# pip3 install -U aws-sam-cli==1.22.0
Requirement already up-to-date: aws-sam-cli==1.22.0 in /usr/local/lib/python3.5/dist-packages
Requirement already up-to-date: jmespath~=0.10.0 in /usr/local/lib/python3.5/dist-packages (from aws-sam-cli==1.22.0)
Requirement already up-to-date: boto3~=1.14 in /usr/local/lib/python3.5/dist-packages (from aws-sam-cli==1.22.0)
Requirement already up-to-date: cookiecutter~=1.7.2 in /usr/local/lib/python3.5/dist-packages (from aws-sam-cli==1.22.0)
Requirement already up-to-date: aws-sam-translator==1.35.0 in /usr/local/lib/python3.5/dist-packages (from aws-sam-cli==1.22.0)
Requirement already up-to-date: aws-lambda-builders==1.3.0 in /usr/local/lib/python3.5/dist-packages (from aws-sam-cli==1.22.0)
Requirement already up-to-date: tomlkit==0.7.0 in /usr/local/lib/python3.5/dist-packages (from aws-sam-cli==1.22.0)
Requirement already up-to-date: dateparser~=0.7 in /usr/local/lib/python3.5/dist-packages (from aws-sam-cli==1.22.0)
Requirement already up-to-date: requests==2.23.0 in /usr/local/lib/python3.5/dist-packages (from aws-sam-cli==1.22.0)
Requirement already up-to-date: watchdog==0.10.3 in /usr/local/lib/python3.5/dist-packages (from aws-sam-cli==1.22.0)
Requirement already up-to-date: PyYAML~=5.3 in /usr/local/lib/python3.5/dist-packages (from aws-sam-cli==1.22.0)
Requirement already up-to-date: serverlessrepo==0.1.10 in /usr/local/lib/python3.5/dist-packages (from aws-sam-cli==1.22.0)
Requirement already up-to-date: Flask~=1.1.2 in /usr/local/lib/python3.5/dist-packages (from aws-sam-cli==1.22.0)
Requirement already up-to-date: chevron~=0.12 in /usr/local/lib/python3.5/dist-packages (from aws-sam-cli==1.22.0)
Requirement already up-to-date: click~=7.1 in /usr/local/lib/python3.5/dist-packages (from aws-sam-cli==1.22.0)
Requirement already up-to-date: docker~=4.2.0 in /usr/local/lib/python3.5/dist-packages (from aws-sam-cli==1.22.0)
Requirement already up-to-date: s3transfer<0.4.0,>=0.3.0 in /usr/local/lib/python3.5/dist-packages (from boto3~=1.14->aws-sam-cli==1.22.0)
Requirement already up-to-date: botocore<1.21.0,>=1.20.50 in /usr/local/lib/python3.5/dist-packages (from boto3~=1.14->aws-sam-cli==1.22.0)
Requirement already up-to-date: binaryornot>=0.4.4 in /usr/local/lib/python3.5/dist-packages (from cookiecutter~=1.7.2->aws-sam-cli==1.22.0)
Requirement already up-to-date: six>=1.10 in /usr/local/lib/python3.5/dist-packages (from cookiecutter~=1.7.2->aws-sam-cli==1.22.0)
Requirement already up-to-date: poyo>=0.5.0 in /usr/local/lib/python3.5/dist-packages (from cookiecutter~=1.7.2->aws-sam-cli==1.22.0)
Requirement already up-to-date: jinja2-time>=0.2.0 in /usr/local/lib/python3.5/dist-packages (from cookiecutter~=1.7.2->aws-sam-cli==1.22.0)
Requirement already up-to-date: MarkupSafe<2.0.0 in /usr/local/lib/python3.5/dist-packages (from cookiecutter~=1.7.2->aws-sam-cli==1.22.0)
Requirement already up-to-date: Jinja2<3.0.0 in /usr/local/lib/python3.5/dist-packages (from cookiecutter~=1.7.2->aws-sam-cli==1.22.0)
Requirement already up-to-date: python-slugify>=4.0.0 in /usr/local/lib/python3.5/dist-packages (from cookiecutter~=1.7.2->aws-sam-cli==1.22.0)
Requirement already up-to-date: jsonschema~=3.2 in /usr/local/lib/python3.5/dist-packages (from aws-sam-translator==1.35.0->aws-sam-cli==1.22.0)
Requirement already up-to-date: setuptools in /usr/local/lib/python3.5/dist-packages (from aws-lambda-builders==1.3.0->aws-sam-cli==1.22.0)
Requirement already up-to-date: wheel in /usr/local/lib/python3.5/dist-packages (from aws-lambda-builders==1.3.0->aws-sam-cli==1.22.0)
Requirement already up-to-date: regex!=2019.02.19 in /usr/local/lib/python3.5/dist-packages (from dateparser~=0.7->aws-sam-cli==1.22.0)
Requirement already up-to-date: pytz in /usr/local/lib/python3.5/dist-packages (from dateparser~=0.7->aws-sam-cli==1.22.0)
Requirement already up-to-date: tzlocal in /usr/local/lib/python3.5/dist-packages (from dateparser~=0.7->aws-sam-cli==1.22.0)
Requirement already up-to-date: python-dateutil in /usr/local/lib/python3.5/dist-packages (from dateparser~=0.7->aws-sam-cli==1.22.0)
Requirement already up-to-date: chardet<4,>=3.0.2 in /usr/local/lib/python3.5/dist-packages (from requests==2.23.0->aws-sam-cli==1.22.0)
Requirement already up-to-date: idna<3,>=2.5 in /usr/local/lib/python3.5/dist-packages (from requests==2.23.0->aws-sam-cli==1.22.0)
Requirement already up-to-date: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.5/dist-packages (from requests==2.23.0->aws-sam-cli==1.22.0)
Requirement already up-to-date: certifi>=2017.4.17 in /usr/local/lib/python3.5/dist-packages (from requests==2.23.0->aws-sam-cli==1.22.0)
Requirement already up-to-date: pathtools>=0.1.1 in /usr/local/lib/python3.5/dist-packages (from watchdog==0.10.3->aws-sam-cli==1.22.0)
Requirement already up-to-date: itsdangerous>=0.24 in /usr/local/lib/python3.5/dist-packages (from Flask~=1.1.2->aws-sam-cli==1.22.0)
Requirement already up-to-date: Werkzeug>=0.15 in /usr/local/lib/python3.5/dist-packages (from Flask~=1.1.2->aws-sam-cli==1.22.0)
Requirement already up-to-date: websocket-client>=0.32.0 in /usr/local/lib/python3.5/dist-packages (from docker~=4.2.0->aws-sam-cli==1.22.0)
Requirement already up-to-date: arrow in /usr/local/lib/python3.5/dist-packages (from jinja2-time>=0.2.0->cookiecutter~=1.7.2->aws-sam-cli==1.22.0)
Requirement already up-to-date: text-unidecode>=1.3 in /usr/local/lib/python3.5/dist-packages (from python-slugify>=4.0.0->cookiecutter~=1.7.2->aws-sam-cli==1.22.0)
Requirement already up-to-date: importlib-metadata; python_version < "3.8" in /usr/local/lib/python3.5/dist-packages (from jsonschema~=3.2->aws-sam-translator==1.35.0->aws-sam-cli==1.22.0)
Requirement already up-to-date: pyrsistent>=0.14.0 in /usr/local/lib/python3.5/dist-packages (from jsonschema~=3.2->aws-sam-translator==1.35.0->aws-sam-cli==1.22.0)
Requirement already up-to-date: attrs>=17.4.0 in /usr/local/lib/python3.5/dist-packages (from jsonschema~=3.2->aws-sam-translator==1.35.0->aws-sam-cli==1.22.0)
Requirement already up-to-date: typing-extensions; python_version < "3.8" in /usr/local/lib/python3.5/dist-packages (from arrow->jinja2-time>=0.2.0->cookiecutter~=1.7.2->aws-sam-cli==1.22.0)
Requirement already up-to-date: zipp>=0.5 in /usr/local/lib/python3.5/dist-packages (from importlib-metadata; python_version < "3.8"->jsonschema~=3.2->aws-sam-translator==1.35.0->aws-sam-cli==1.22.0)
You are using pip version 8.1.1, however version 21.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
root@eb3dc6ff9997:/localDebugRepo/infrastructure# python --version
bash: /usr/bin/python: No such file or directory
root@eb3dc6ff9997:/localDebugRepo/infrastructure# pip --version
bash: /usr/bin/pip: No such file or directory
root@eb3dc6ff9997:/localDebugRepo/infrastructure# python3 --version
Python 3.5.2
root@eb3dc6ff9997:/localDebugRepo/infrastructure# pip3 --version
pip 8.1.1 from /usr/lib/python3/dist-packages (python 3.5)
root@eb3dc6ff9997:/localDebugRepo/infrastructure# sam --version
Traceback (most recent call last):
  File "/usr/local/bin/sam", line 7, in <module>
    from samcli.cli.main import cli
  File "/usr/local/lib/python3.5/dist-packages/samcli/cli/main.py", line 86
    cmdline_args_log += f"--{key} "
                                  ^
SyntaxError: invalid syntax
root@eb3dc6ff9997:/localDebugRepo/infrastructure# 

I would really like there to be some sane way to know exactly which Python version I need and a sane way to install and configure it.

Please help me:)

Answer 1

In most cases, instead of modifying the python installation in an existing image or installing a new one, I would recommend just using a docker image that already has python 3 installed. This bitbucket page talks about using different base images.

There's also a page about using a python base image spectifically .

Aside from this, depending on how you want to use SAM, you may be able to use the builtin Bitbucket "pipe" for deploying to SAM instead. Here is some info on pipes. . I think this would bypass the need to install SAM in your build.

I haven't used that particular pipe myself but the way I think it works is you build your deploy artifact file in your pipeline, caching it so it's available in future steps, then you set that file as the input for the SAM pipe to deploy it. The SAM pipe runs in a separate container, so it's already installed in that container. You just pass in the file.

Finally, I have never used SAM, so take this with a grain of salt, but I don't see a reference to installing the CLI with pip either on the official AWS Linux installation docs or on the github page . I did find this issue , from which I glean the following:

• pip installation is possible but not recommended
• Python 3.5 is not supported by aws sam cli
• If you use the installer in the instructions, it sounds like it will take care of the python versioning for you.
• They would like to target a single version of Python, so if you use pip, maybe the latest or 2nd latest Python version will work. Just a guess...