Connecting to a Cloud SQL instance from VM with public and private IPs - how to ensure the right network interface is chosen for the connection?

Question

What I'm trying to set up:

• Cloud SQL instance with private IP, Postgresql database
• A VM with a public IP, but also one private IP on same VPC network as the SQL instance is on (VM, SQL instance and VPC are all in the same region)
• VM has a service account with sufficient Cloud SQL client/viewer permissions
• Possibility to connect from VM to SQL instance.

What happens?

• Any attempt to actually use the connection, from for example psql client or db-migrate, simply hangs - for example psql --host 10.78.0.3 -U gcp-network-issue-demo-staging-db-user gcp-network-issue-demo-staging-database will not prompt for a password, just sit there.
• If I remove the VM's public IP address from the setup, it connects fine. However, I need a publicly accessible VM for other services to connect to it..

I assume the psql connection attempt goes through the wrong network interface or something (this may be just my ignorance about network stuff speaking) - how can I get this working? What am I missing?

PS: this is basically same problem as Connecting to Google Cloud SQL instance on private IP from a VM with both private and public IPs fails but commenters there seem to want one Terraform-related and one connection-issue-related question.

Some screenshots:

VM IPs:

DB IPs:

Network config for VM:

Private IP config for DB instance:

This is the setup of the private network:

I don't understand why the private IP of the DB instance ( 10.78.0.3 ) is not an IP from the range of the private network ( 10.2.0.0 - 10.2.0.24 , right?)..? Is that my problem?

Answer 1

To answer your question:

I don't understand why the private IP of the DB instance (10.78.0.3) is not an IP from the range of the private network (10.2.0.0-10.2.0.24, right?)..?

The Cloud SQL instance is assigned an IP address from the allocated range . When you setup a private services access a VPC peering is created between your VPC gcp-network-issue-demo-staging-network and the service producer VPC network that uses the allocated range 10.78.0.0/16

Also, looking at your VM network config, I see that the VM has two Nics in two different VPCs ( default and gcp-network-issue-demo-staging-network ). In your case, you can use only one Nic.

As a next step, make sure that your VM is using only the VPC network that you have used to create the private connection . Once that done you should be able to connect to the Cloud SQL instance IP using the command bellow:

telnet 10.78.0.3 3306