Python script to store Json files in real time to amazon S3

Question

I have a python code that gives me tweets in real time using Twitter Streaming API. I have stored the output to a json file which keeps on updating dynamically as new tweets arrive.However, I would like to save this json to amazon s3 which I could use to trigger events using amazon lambda service.Can somebody suggest me a way to solve this problem?

Answer 1

Amazon has a python library to interact with it's web services. It's called Boto and supports both S3 and lambda services.

To save the json to S3:

import boto3

s3 = boto3.resource('s3')
s3.create_bucket(Bucket='jsonbucket')
s3.Object('jsonbucket', 'yourstoredjson.json').put(Body=open('yourlocaljson.json', 'rb'))

Note: before saving files to S3 with boto, you have to set up authentication credentials for AWS.