如何使用 boto3 从 S3 向 RDS 写入数据

Question

I want to save files that are put in my S3 bucket every 2 minutes. I want to move this data to RDS using python. I am wondering what is the equivalence of this code for saving it to DynamoDB in RDS. I cannot find an equivalent for the batch_writer boto3 DynamoDB call.

Thanks in advance! If you know a better approach feel free to comment.

import boto3
import csv
import os
import tempfile

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('Movies')
s3 = boto3.client('s3')


def lambda_handler(event, context):

for record in event['Records']:
    source_bucket = record['s3']['bucket']['name']
    key = record['s3']['object']['key']
    with tempfile.TemporaryDirectory() as tmpdir:
        download_path = os.path.join(tmpdir, key)
        s3.download_file(source_bucket, key, download_path)
        items = read_csv(download_path)

        with table.batch_writer() as batch:
            for item in items:
                batch.put_item(Item=item)

def read_csv(file):
    items = []
    with open(file) as csvfile:
        reader = csv.DictReader(csvfile)
        for row in reader:
            data = {}
            data['Meta'] = {}
            data['Year'] = int(row['Year'])
            data['Title'] = row['Title'] or None
            data['Meta']['Length'] = int(row['Length'] or 0)
            data['Meta']['Subject'] = row['Subject'] or None
            data['Meta']['Actor'] = row['Actor'] or None
            data['Meta']['Actress'] = row['Actress'] or None
            data['Meta']['Director'] = row['Director'] or None
            data['Meta']['Popularity'] = row['Popularity'] or None
            data['Meta']['Awards'] = row['Awards'] == 'Yes'
            data['Meta']['Image'] = row['Image'] or None
            data['Meta'] = {k: v for k,
                            v in data['Meta'].items() if v is not None}
            items.append(data)
    return items

Answer 1

You can't write to RDS using Boto3, unless you are running Aurora Serverless. You would need to use the database connection library for Python that corresponds to the RDBMS engine (MySQL, PostgreSQL, etc.) that you are running in RDS. You would perform batch inserts using the SQL INSERT statement.

Add more details about your RDS database to your question if you need a more detailed answer.