Execute python script on EC2 instance via AWS lambda

Question

I have an AWS EC2 instance on linux running 24/7 with a set of python files that are working properly. From the shell, I can input python python_script.py var1 var2 . var1 and var2 are simply integers with values under 5,000.

I am able to set up the AWS API to receive HTTPS requests that deliver these variables as environmental variables. Then I can use AWS lambda to run some code when the API is triggered.

I can't for the life of me figure out what code to include in the lambda function in order to run the python script on the EC2 instance with the two variables. All of the results that I have found related to this issue are related to scripts on startup of an EC2 instance, run commands without any variables, or setting up an entire web server with domain and everything to receive the HTTPS request. I simply want to be able to access the API from a remote location, deliver two integers, then run the python file with those integers.

I did find advice on How to run a script on your aws-ec2 from a script on your local machine?

ssh -i key.pem ubuntu@ec2-instance "bash /path/to/your/script/data.sh"

So for my own use case, I have tried modifying it to:

ssh -i python_bot.pem ubuntu@ec2-instance "python Python_Project/python_script.py 2 166"

However, when I run this code from my own shell, I get bash: python: command not found . This is strange, because if I do ssh -i python_bot.pem ubuntu@ec2-instance and then once logged in do python Python_Project/python_script.py 2 166 the file runs successfully. I am guessing if I can get this to work, then I can try some solutions from Can bash script be written inside a AWS Lambda function

I have also found Running Python Script in an existing EC2 instance on AWS and while the answer gave me some direction, it is too broad to help with a specific solution.

Answer 1

Trigger a script on an Amazon EC2 instance from an AWS Lambda function is not a good architectural design.

As described in Running Python Script in an existing EC2 instance on AWS (to which you linked), a computer doesn't generally like things being "pushed" to it, unless it is running as a web server.

Instead, I would like to suggest to you a different architecture :

• The Lambda function pushes a 'job' onto an Amazon SQS queue
• Code on the Amazon EC2 instance regularly checks ('polls') the SQS queue to see whether any messages are available
• If a message is available, it uses the content of the message to trigger some local work on the EC2 instance

This way, the two systems are disconnected, but are communicating via the SQS queue. It has the additional benefit that, if the EC2 instance is busy or offline, the messages (jobs) accumulate in the SQS queue until they can be processed.

This architecture is better than attempting to use SSH or SCP to trigger a job (which is not a very 'cloud' way to architect a system).

Answer 2

Login shells and running commands via SSH may have different environment variables, especially PATH here. Being explicit about the interpreter should help.

Log in to the instance and see which Python you're using:

which python

Then substitute that into your SSH command line, eg

ssh -i python_bot.pem ubuntu@ec2-instance "/usr/bin/python Python_Project/python_script.py 2 166"