private IP address range for GCP Cloud SQL is ignored

Question

I've been trying to set up Google Cloud SQL with a private IP connection, where the IP range it's bound to is manually allocated, and have not succeeded. I don't know if this is a bug in the implementation because it's still in beta, if there's something missing from the docs, or if I'm just doing something wrong. (A command-line session is at the bottom, for a quick summary of what I'm seeing.)

Initially, I set it up to automatically allocate the IP range. It all worked just fine, except that it chose 172.17.0.0/24, which is one of the networks managed by docker on my GCE instance, so I couldn't connect from there (but could on another machine without docker). So then I tried going down the manual allocation route.

First, I tore down all the associated network objects that had been created on my behalf. There were two VPC Peerings, cloudsql-postgres-googleapis-com and servicenetworking-googleapis-com , which I deleted, and then I confirmed that the routing entry associated with them disappeared as well.

Then, I followed the directions at https://cloud.google.com/vpc/docs/configure-private-services-access#allocating-range , creating 10.10.0.0/16, because I wanted it in my default network, which is auto mode, so I'm limited to the low half (which is currently clear).

At that point, I went back to the Cloud SQL instance creation page, since it should be doing the rest for me. I checked the "Private IP" box, and chose the default network.

I wasn't taking notes at the time, so my recollection may be flawed, particularly since my experience in later attempts was consistently different, but what I remember seeing was that below the network choice dropdown, it said "This instance will use the existing managed service connection". I assumed that meant it would use the address range I'd created, and went forward with the instance creation, but the instance landed on the 172.17.0.0/24 network again.

Back around the third time, where that message was before, it had a choice box listing my address range. Again, my recollection was poor, so I don't know if I either saw or clicked on the "Connect" button, but the end result was the same.

On the fourth attempt, I did notice the "Connect" button, and made sure to click it, and wait for it to say it succeeded. Which it did, sort of: it replaced the dropdown and buttons with the same message I'd seen before about using the existing connection. And again, the instance was created on the wrong network.

I tried a fifth time, this time having created a new address range with a new name -- google-managed-services-default -- which was the name that the automatic allocation had given it back when I first started (and what the private services access docs suggest). But even with that name, and explicitly choosing it, I still ended up with the instance on the wrong network.

Indeed, I now see that after I click "Connect", I can go check the routes and see that the route that was created is to 172.17.0.0/24.

The same thing seems to happen if I do everything from the command-line:

$ gcloud beta compute addresses list
NAME                             ADDRESS/RANGE    TYPE      PURPOSE      NETWORK  REGION    SUBNET  STATUS
google-managed-services-default  10.11.0.0/16     INTERNAL  VPC_PEERING  default                    RESERVED
$ gcloud beta services vpc-peerings connect \ 
    --service=servicenetworking.googleapis.com \
    --ranges=google-managed-services-default \
    --network=default \
    --project=...
$ gcloud beta services vpc-peerings list --network=default
--- 
network: projects/.../global/networks/default
peering: servicenetworking-googleapis-com
reservedPeeringRanges: 
- google-managed-services-default
---
network: projects/.../global/networks/default
peering: cloudsql-postgres-googleapis-com
reservedPeeringRanges:
- google-managed-services-default
$ gcloud beta compute routes list
NAME                            NETWORK      DEST_RANGE     NEXT_HOP                          PRIORITY
peering-route-ad7b64a0841426ea  default      172.17.0.0/24  cloudsql-postgres-googleapis-com  1000

So now I'm not sure what else to try. Is there some state I didn't think to clear? How is the route supposed to be connected to the address range? Why is it creating two peerings when I only asked for one? If I were to create a route manually to the right address range, I presume that wouldn't work, because the Postgres endpoint would still be at the wrong address.

(Yes, I could reconfigure docker, but I'd rather not.)

Answer 1

I found here https://cloud.google.com/sql/docs/mysql/private-ip that this seems to be the correct behaviour:

After you have established a private services access connection, and created a Cloud SQL instance with private IP configured for that connection, the corresponding (internal) subnet and range used by the Cloud SQL service cannot be modified or deleted. This is true even if you delete the peering and your IP range. After the internal configuration is established, any Cloud SQL instance created in that same region and configured for private IP uses the original internal configuration.

Answer 2

There turned out to be a bug somewhere in the service machinery of Google Cloud, which is now fixed. For reference, see the conversation at https://issuetracker.google.com/issues/118849070 .