从 VPC 将 Glue 连接到 RDS、S3 和 Secrets Manager

Question

I need my Glue job to access a VPC accessible RDS, s3 bucket and Secrets Manager.

I've created a JDBC connection and verified that it works fine. I've also created a.network connection and verified that it has access to the required s3 bucket. These were both confirmed using the "Test connection" feature in the AWS UI.

When I create a Glue job, I can add multiple connections. I add both connections.

If I add the.network connection first, I can successfully connect to s3 but the connection to SecretsManager times out. If I add the JDBC connection first the s3 connection fails but the SecretsManager and RDS connection work fine.

Is there something I'm missing in terms of this configuration? Are multiple connections actually supported or it a UI bug that I can add multiple?

Thanks for any pointers.

Answer 1

I was pretty ignorant of Glue when I asked above.

I only needed a single connection, of type "Network" to allow access to the 3 resources that I outlined above.

Once this connection was correctly configured (and by correctly I mean Security Groups and ACL access were in place) I could access:

• my RDS (in another private su.net in the VPC)
• S3 (via correctly configured VPC Gateway)
• SecretsManager (via correctly configured VPC Gateway)

IMVHO, the "JDBC connection" feature is a little confusing - at least it was for me. It is *not * required to access a Database. It is a "special" type of Network connection which can be used to store credential info as metadata on the connection object (check with your security folks before you do this - I don't know how this is stored on the AWS backend).

Anyway, Glue wasn't the issue here - my own ignorance was.