Could not connect Cloud SQL PostgreSQL from Data Fusion

Question

Trying to connect a Cloud SQL PotgreSQL database from a Cloud Data Fusion (both private instances in the same VPC, not shared) as described here step by step:

https://cloud.google.com/data-fusion/docs/how-to/reading-from-postgresql https://cloud.google.com/data-fusion/docs/how-to/create-private-ip

With this postgres-socket-factory-1.3.0-jar-with-driver-and-dependencies.jar as driver, downloaded from: https://github.com/GoogleCloudPlatform/cloud-sql-jdbc-socket-factory/releases/tag/v1.3.0

Testing the connection, wrangler gets broken like this:

wrangler broken

But also Data Fusion Wranger logs describes this error:

2021-06-29 21:28:02,226 - WARN  [service-http-executor-1147:i.c.w.s.c.AbstractWranglerHandler@210] - Error processing POST /v3/namespaces/system/apps/dataprep/services/service/methods/contexts/default/connections/databases, resulting in a 500 response.
java.lang.ExceptionInInitializerError: null
    at com.google.api.services.sqladmin.SQLAdmin$Builder.build(SQLAdmin.java:7339) ~[na:na]
    at com.google.cloud.sql.core.CoreSocketFactory.createAdminApiClient(CoreSocketFactory.java:365) ~[na:na]
    at com.google.cloud.sql.core.CoreSocketFactory.getInstance(CoreSocketFactory.java:141) ~[na:na]
    at com.google.cloud.sql.core.CoreSocketFactory.connect(CoreSocketFactory.java:252) ~[na:na]
    at com.google.cloud.sql.postgres.SocketFactory.createSocket(SocketFactory.java:76) ~[na:na]
    at org.postgresql.core.PGStream.createSocket(PGStream.java:223) ~[na:na]
    at org.postgresql.core.PGStream.<init>(PGStream.java:95) ~[na:na]
    at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:98) ~[na:na]
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:213) ~[na:na]
    at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:51) ~[na:na]
    at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:223) ~[na:na]
    at org.postgresql.Driver.makeConnection(Driver.java:465) ~[na:na]
    at org.postgresql.Driver.connect(Driver.java:264) ~[na:na]
    at io.cdap.wrangler.service.database.JDBCDriverShim.connect(JDBCDriverShim.java:44) ~[expanded.16249867015644.4.1.86000cc8-2950-4df3-8ded-ef6513642958.jar/:na]
    at java.sql.DriverManager.getConnection(DriverManager.java:664) ~[na:1.8.0_292]
    at java.sql.DriverManager.getConnection(DriverManager.java:247) ~[na:1.8.0_292]
    at io.cdap.wrangler.service.database.DatabaseHandler.loadAndExecute(DatabaseHandler.java:558) [expanded.16249867015644.4.1.86000cc8-2950-4df3-8ded-ef6513642958.jar/:na]
    at io.cdap.wrangler.service.database.DatabaseHandler.lambda$listDatabases$4(DatabaseHandler.java:322) [expanded.16249867015644.4.1.86000cc8-2950-4df3-8ded-ef6513642958.jar/:na]
    at io.cdap.wrangler.service.common.AbstractWranglerHandler.respond(AbstractWranglerHandler.java:200) ~[expanded.16249867015644.4.1.86000cc8-2950-4df3-8ded-ef6513642958.jar/:na]
    at io.cdap.wrangler.service.database.DatabaseHandler.listDatabases(DatabaseHandler.java:316) [expanded.16249867015644.4.1.86000cc8-2950-4df3-8ded-ef6513642958.jar/:na]
    at io.cdap.wrangler.service.database.DatabaseHandlercdb75ebbd71058e7f79bb5e06b7f47ea.lambda$listDatabases_Lio_cdap_cdap_api_service_http_HttpServiceRequest_Lio_cdap_cdap_api_service_http_HttpServiceResponder_Ljava_lang_String__V(Unknown Source) [na:na]
    at io.cdap.cdap.internal.app.runtime.AbstractContext.execute(AbstractContext.java:560) ~[na:na]
    at io.cdap.cdap.internal.app.services.ServiceHttpServer$HandlerDelegatorContext$1.execute(ServiceHttpServer.java:230) ~[na:na]
    at io.cdap.cdap.internal.app.runtime.service.http.AbstractHttpHandlerDelegator.execute(AbstractHttpHandlerDelegator.java:66) ~[na:na]
    at io.cdap.wrangler.service.database.DatabaseHandlercdb75ebbd71058e7f79bb5e06b7f47ea.listDatabases(Unknown Source) [na:na]
    at sun.reflect.GeneratedMethodAccessor128.invoke(Unknown Source) ~[na:na]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_292]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_292]
    at io.cdap.http.internal.HttpMethodInfo.invoke(HttpMethodInfo.java:87) ~[io.cdap.http.netty-http-1.5.0.jar:na]
    at io.cdap.http.internal.HttpDispatcher.channelRead(HttpDispatcher.java:45) ~[io.cdap.http.netty-http-1.5.0.jar:na]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[io.netty.netty-all-4.1.16.Final.jar:4.1.16.Final]
    at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:38) ~[io.netty.netty-all-4.1.16.Final.jar:4.1.16.Final]
    at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:353) ~[io.netty.netty-all-4.1.16.Final.jar:4.1.16.Final]
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) ~[io.netty.netty-all-4.1.16.Final.jar:4.1.16.Final]
    at io.cdap.http.internal.NonStickyEventExecutorGroup$NonStickyOrderedEventExecutor.run(NonStickyEventExecutorGroup.java:254) ~[io.cdap.http.netty-http-1.5.0.jar:na]
    at io.netty.util.concurrent.UnorderedThreadPoolEventExecutor$NonNotifyRunnable.run(UnorderedThreadPoolEventExecutor.java:277) ~[io.netty.netty-all-4.1.16.Final.jar:4.1.16.Final]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_292]
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_292]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[na:1.8.0_292]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[na:1.8.0_292]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_292]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_292]
    at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_292]
Caused by: java.lang.IllegalStateException: You are currently running with version 1.30.9 of google-api-client. You need at least version 1.31.1 of google-api-client to run version 1.31.0 of the Cloud SQL Admin API library.
    at com.google.common.base.Preconditions.checkState(Preconditions.java:537) ~[guava-20.0.jar:na]
    at com.google.api.client.util.Preconditions.checkState(Preconditions.java:113) ~[google-http-client-1.34.2.jar:1.34.2]
    at com.google.api.services.sqladmin.SQLAdmin.<clinit>(SQLAdmin.java:44) ~[na:na]
    ... 43 common frames omitted

Notice the Caused by: java.lang.IllegalStateException: You are currently running with version 1.30.9 of google-api-client. You need at least version 1.31.1 of google-api-client to run version 1.31.0 of the Cloud SQL Admin API library. Caused by: java.lang.IllegalStateException: You are currently running with version 1.30.9 of google-api-client. You need at least version 1.31.1 of google-api-client to run version 1.31.0 of the Cloud SQL Admin API library.

Update: Also tried with a genererated fat-jar from sources with same result.

Testing conection with same driver from pipeline Studio I get this error:

2021-06-30 08:39:25,091 - ERROR [studio-http-executor-49:i.c.p.d.b.s.AbstractDBSource@140] - Exception while performing getSchema
org.postgresql.util.PSQLException: The connection attempt failed.
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:315) ~[na:na]
    at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:51) ~[na:na]
    at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:223) ~[na:na]
    at org.postgresql.Driver.makeConnection(Driver.java:465) ~[na:na]
    at org.postgresql.Driver.connect(Driver.java:264) ~[na:na]
    at io.cdap.plugin.db.JDBCDriverShim.connect(JDBCDriverShim.java:60) ~[database-commons-1.5.3.jar:na]
    at java.sql.DriverManager.getConnection(DriverManager.java:664) ~[na:1.8.0_292]
    at java.sql.DriverManager.getConnection(DriverManager.java:208) ~[na:1.8.0_292]
    at io.cdap.plugin.db.batch.source.AbstractDBSource.getConnection(AbstractDBSource.java:219) [database-commons-1.5.3.jar:na]
    at io.cdap.plugin.db.batch.source.AbstractDBSource.getSchema(AbstractDBSource.java:132) [database-commons-1.5.3.jar:na]
    at io.cdap.plugin.db.batch.source.AbstractDBSource.configurePipeline(AbstractDBSource.java:113) [database-commons-1.5.3.jar:na]
    at io.cdap.plugin.cloudsql.postgres.CloudSQLPostgreSQLSource.configurePipeline(CloudSQLPostgreSQLSource.java:57) [1625042353080-0/:na]
    at io.cdap.cdap.etl.spec.PipelineSpecGenerator.configureStage(PipelineSpecGenerator.java:306) [cdap-etl-core-6.4.1.jar:na]
    at io.cdap.cdap.datapipeline.service.ValidationHandler.validateStage(ValidationHandler.java:161) [expanded.16249866968976.4.1.73eaa383-b419-44a0-be67-36b2647bbaa0.jar/:na]
    at io.cdap.cdap.datapipeline.service.ValidationHandler4b0e8282e7c248900dbfd7c39c0c3cc8.lambda$validateStage_Lio_cdap_cdap_api_service_http_HttpServiceRequest_Lio_cdap_cdap_api_service_http_HttpServiceResponder_Ljava_lang_String__V(Unknown Source) [na:na]
    at io.cdap.cdap.internal.app.services.ServiceHttpServer$HandlerDelegatorContext$1.lambda$execute$0(ServiceHttpServer.java:228) ~[na:na]
    at io.cdap.cdap.internal.app.runtime.AbstractContext.lambda$execute$3(AbstractContext.java:515) ~[na:na]
    at io.cdap.cdap.data2.transaction.Transactions$CacheBasedTransactional.finishExecute(Transactions.java:224) ~[na:na]
    at io.cdap.cdap.data2.transaction.Transactions$CacheBasedTransactional.execute(Transactions.java:211) ~[na:na]
    at io.cdap.cdap.internal.app.runtime.AbstractContext.execute(AbstractContext.java:512) ~[na:na]
    at io.cdap.cdap.internal.app.runtime.AbstractContext.execute(AbstractContext.java:500) ~[na:na]
    at io.cdap.cdap.internal.app.services.ServiceHttpServer$HandlerDelegatorContext$1.execute(ServiceHttpServer.java:228) ~[na:na]
    at io.cdap.cdap.internal.app.runtime.service.http.AbstractHttpHandlerDelegator.execute(AbstractHttpHandlerDelegator.java:66) ~[na:na]
    at io.cdap.cdap.datapipeline.service.ValidationHandler4b0e8282e7c248900dbfd7c39c0c3cc8.validateStage(Unknown Source) [na:na]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_292]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_292]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_292]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_292]
    at io.cdap.http.internal.HttpMethodInfo.invoke(HttpMethodInfo.java:87) ~[io.cdap.http.netty-http-1.5.0.jar:na]
    at io.cdap.http.internal.HttpDispatcher.channelRead(HttpDispatcher.java:45) ~[io.cdap.http.netty-http-1.5.0.jar:na]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[io.netty.netty-all-4.1.16.Final.jar:4.1.16.Final]
    at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:38) ~[io.netty.netty-all-4.1.16.Final.jar:4.1.16.Final]
    at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:353) ~[io.netty.netty-all-4.1.16.Final.jar:4.1.16.Final]
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) ~[io.netty.netty-all-4.1.16.Final.jar:4.1.16.Final]
    at io.cdap.http.internal.NonStickyEventExecutorGroup$NonStickyOrderedEventExecutor.run(NonStickyEventExecutorGroup.java:254) ~[io.cdap.http.netty-http-1.5.0.jar:na]
    at io.netty.util.concurrent.UnorderedThreadPoolEventExecutor$NonNotifyRunnable.run(UnorderedThreadPoolEventExecutor.java:277) ~[io.netty.netty-all-4.1.16.Final.jar:4.1.16.Final]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_292]
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_292]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[na:1.8.0_292]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[na:1.8.0_292]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_292]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_292]
    at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_292]
Caused by: java.net.SocketTimeoutException: connect timed out
    at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_292]
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[na:1.8.0_292]
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[na:1.8.0_292]
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_292]
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_292]
    at java.net.Socket.connect(Socket.java:607) ~[na:1.8.0_292]
    at org.postgresql.core.PGStream.createSocket(PGStream.java:231) ~[na:na]
    at org.postgresql.core.PGStream.<init>(PGStream.java:95) ~[na:na]
    at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:98) ~[na:na]
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:213) ~[na:na]
    ... 42 common frames omitted

Answer 1

Using version 1.2 and below of postgres-socket-factory will fix Google API library version issue as confirmed by @Anthanh Pham Trinh.

The wrangler test connection issue as mentioned in the question is a known issue and fix for this can be tracked in https://cdap.atlassian.net/browse/CDAP-17973 .

---

EDIT: 20210729

To use a different driver rather than the default v1.0.16 that Data Fusion let's you download. You can download the different versions of cloud-sql-jdbc drivers in this github link(version 1.2 included) .

When the version 1.2 driver is downloaded:

• Open Cloud Data Fusion Instance

• Click "HUB"

• Click "Drivers"

• Search for "CloudSQL PostgreSQL JDBC Driver" select it

  

• Click "Deploy"

  

• Upload the driver that you have downloaded in cloud-sql-jdbc drivers github link . In this example I downloaded postgres-socket-factory-1.2.0-jar-with-driver-and-dependencies.jar

  

• Click "Next", driver configuration will be auto populated since it will use the details of the uploaded driver.

  

• Click "Finish"

• When upload is done, it will show the driver version that was uploaded.

  

You can also check this by going to "Wrangler"

• When in "Wrangler", click "+ Add Connection"

• Click "Database"

  

• This will show all the available database connections, and PostgreSQL version 1.2 should be available.