Can we use AWS glue for analysing the RDS database and store the analysed data into rds mysql table using ETL

Question

I am new in AWS. I want to use AWS glue for ETL process.

Could we use AWS glue for analyzing the RDS database and store the analyzed data into rds mysql table using ETL job

Thanks

Answer 1

Yes, its possible. We have used S3 to store our raw data, from where we read the data in AWS Glue, and perform UPSERTs to RDS Aurora as part of our ETL process. You can either use AWS Glue trigger or a Lambda S3 event triggers for calling the glue job.

We have used pymysql / mysql.connector in AWS Glue since we have to do UPSERTs. Bulk load data directly from S3 is also supported for RDS Mysql (Aurora). Let me know if you need help with code sample