AWS Data Pipeline将CSV从S3复制到RDS MySQL

Question

I have a directory within my S3 bucket that contains many .CSV files that are all formatted the same way (First, Last, Location, Date).

I have been trying to use Data Pipeline to populate an RDS MySQL Database table with the contents of these CSV files. Fortunately, Amazon provides a template for this action already.

"Load S3 data into RDS MySQL table" http://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-template-copys3tords.html

I have filled out all of the appropriate information that it requests.

When I activate the pipeline, it creates the CopyActivity and the ShellCommandActivity. Copy activity copies the data, and Shell command activity creates the table if it thinks it isn't there already. The Shell command activity successfully connects to my RDS.

However, my issue is that the ShellCommandActivity switches to "FINISHED" status without actually creating a table, and then the CopyActivity gets stuck at "WAITING_ON_DEPENDENCIES". This whole process takes around 20 minutes.

All of my roles have full access to all of the services.

If anyone has any insight, please comment. I have been stuck on this issue for nearly 2 weeks now.

Answer 1

我尝试使用SQLActivity选择要创建的表中的所有数据，这个新的SQLActivity将是CopyActivity的依赖项。