如何将 AWS Glue 与文档数据库连接

Question

Do anyone aware how to connect Glue with Document DB?

Unfortunately the AWS blog [https://aws.amazon.com/blogs/big-data/building-aws-glue-spark-etl-jobs-using-amazon-documentdb-with-mongodb-compatibility-and-mongodb/] i followed is not working as expected.

Steps i followed

• Created the Doc db Security group to open the port 27017
• Created a Doc DB using my default VPC in personal account
• Able to connect to DocumentDb using Cloud9 and created a document in test db
• Created a Connection of Document DB in Glue Data Catalogue [while creating chosen default vpc security group and Doc db security group]
• Created a S3 VPC endpoint
• Created a glue VPC endpoint
• Create a Job using script given in blog but it's failing in line [dynamic_frame2 = glueContext.create_dynamic_frame.from_options]

Error - An error occurred while calling o92.getDynamicFrame. Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=<cluster>:<port>, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.SocketTimeoutException: connect timed out}}]

Answer 1

I figured it out. Since this job was created using Spark script editor, DB connection needs to be in place. Open the Job Script, go to Job Details tab click advanced properties and selected the Doc Db connection from the drop down and it worked.