MySQL Aurora and AWS S3: Need an alternate way of MySQL's "LOAD DATA" for loading document data from S3

Question

I need to import data from files stored in S3 into an MySQL Aurora db.

I have Eventbridge setup so when the file is added to S3 it fires an event that calls a lambda.

The lambda needs to import the file data into MySQL. The MySQL "LOAD DATA FROM S3" feature would be great for this..... but.... you will get the error: This command is not supported in the prepared statement protocol yet .

LOAD DATA has a lot of limitations such as this, it cannot be be in a stored procedure, cannot be in dynamic SQL (really needed here). I cannot find a hack work-around for this and need an alternate way to import data directly from S3 to MySQL. I don't want to move the data from S3 to Lambda to MySQL as that extra step in the middle adds a lot of exposure for failure.

Does anyone know any good ideas (and even not so good) for moving data from S3 to MySQL Aurora?

Thanks.

Answer 1

1. Check whether the user has permission to import data, which requires AWS_LOAD_S3_ACCESS permission
2. Examples of import statements can be referred to as follows:

LOAD DATA FROM S3 's3://mybucket/data.txt'
     INTO TABLE table1
     (column1, column2)
     SET column3 = CURRENT_TIMESTAMP;

3. For more detailed information, please refer to https://docs.amazonaws.cn/en_us/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Integrating.LoadFromS3.html