每当触发 lambda 时，如何运行 AWS EC2 中存在的 python 脚本？

Question

I have a python script present in my AWS EC2 instance that does some job. I have to trigger that script whenever a new file enters a particular bucket.

My idea was to add a lambda trigger to that bucket which in turns triggers the script present in EC2, but failed to do so.

So how to achieve the solution if according to my plan or is there any other workarounds for this problem?

Answer 1

As suggested in comment better to use SNS or SQS , I think it more suitable then lambda function and with SNS or SQS involve one to one communication between S3 and EC2 instance then why you should add an extra layer of lambda?.

Although three can subscribe to the event but lambda involves one extra layer and also involve ssh too which I think costly in term of time (s3 event receiving + event process + ssh to ec2).

Using Lambda:

When lambda trigger it will start doing ssh to ec2 and will run the script and there is one big advantage with Lambda is that you can run any type of script and you do not need a server to keep them up and running like in case of SQS and SNS. you can explore these example ssh-ec2-lambda/ and scheduling-ssh-jobs-using-aws-lambda , the second example is similar just you need based on the event instead of scheduling.

SNS:

If multiple instances suppose to run the job script on ec2 instance the SNS is a better choice. The diagram is a bit similar to your use cases or for representing a big picture.

SQS:

If there is only one instance is supposed to run the script then SQS will be suitable to handle the event.

Answer 2

• I am not sure why your option did not work, because its absolutely possible and i have done this using this blog aws blog
• This git repository has code to trigger a lambda whenever an file with a specific extension is uploaded to the bucket (terraform).
• You can access the EC2 instance via the lambda as shown in the block above using tags.
• Hope this all helps you.

Answer 3

I managed it with the help of a blog I found online, which I have lost the link of, but have the code.

import time
import boto3
import paramiko
import os

def lambda_handler(event, context):

    ec2 = boto3.resource('ec2', region_name='us-east-1',aws_access_key_id='XXXXXXXXXXXXXXXXXXXX',aws_secret_access_key='XXXXXXXXXXXXXXXXXXXX')

    instance_id = 'XXXXXXXXXXXXXXXX'

    instance = ec2.Instance(instance_id)

    # Start the instance
    instance.start()

    # Giving some time to start the instance completely
    #time.sleep(60)

    # Connect to S3, we will use it get the pem key file of your ec2 instance
    s3_client = boto3.client('s3',aws_access_key_id='XXXXXXXXXXXXXXXXXXXX',aws_secret_access_key='XXXXXXXXXXXXXXXXXXXX')

    # # # Download private key file from secure S3 bucket
    # # # and save it inside /tmp/ folder of lambda event
    bucket_name = ''
    key_name = ''
    key_location = ''
    s3_client.download_file(bucket_name, key_name, key_location)

    # # # # Allowing few seconds for the download to complete
    time.sleep(10)

    ssh = paramiko.SSHClient()
    ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    privkey = paramiko.RSAKey.from_private_key_file(key_location)
    # # # username is most likely 'ec2-user' or 'root' or 'ubuntu'
    # # # depending upon yor ec2 AMI
    ssh.connect(instance.private_ip_address,22, username='ubuntu', pkey=privkey)



    commands = []
    for command in commands:
        print("Executing {}".format(command))
        stdin , stdout, stderr = ssh.exec_command(command)
        stdin.flush()
        data = stdout.read().splitlines()
        for line in data:
            print(line)
    ssh.close()

    return 'Success'

Now just zip the paramiko library along with it. Will udate the answer if I found the blog again.