无法从 Data Fusion 连接 Cloud SQL mySql/postgreSQL 实例

Question

Goal is to connect Cloud SQL mysql or postgreSQL instances using Cloud Data Fusion.

• created Cloud SQL instances with MySQL and postgreSQL
• created Cloud Data Fusion instance
• From wrangler > Add connection > Cloud SQL MySQL
• Added Data Fusion instance as member in IAM and added permissions to following Cloud SQL Client Cloud Data Fusion API Service Agent
• In add connection used jdbc url as jdbc:mysql://google/mysql?cloudSqlInstance=&socketFactory=com.google.cloud.sql.mysql.SocketFactory&useSSL=false
• Added driver for mySQL: mysql-connector-java-5.1.39-bin.jar
• Added MySQL socket factory jar as library.

While testing connection, it fails with error: com.mysql.jdbc.Driver

Expecting testing of the connection with Cloud SQL MySQL to be successful so that the data fusion pipeline can be built.

Answer 1

The issue is that you need to provide a jar that has both the driver and the connector included. You can find instructions on how to build a uberjar (also called a farjar) using the driver and JDBC Socket Factory from this post here .

Edit: Here is a POM for MySQL:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
   <modelVersion>4.0.0</modelVersion>
   <groupId>com.example</groupId>
   <artifactId>mysql-with-cloud-sql-socket-factory</artifactId>
   <packaging>pom</packaging>
   <version>0.0.1</version>

   <dependencies>
      <dependency>
         <groupId>mysql</groupId>
         <artifactId>mysql-connector-java</artifactId>
         <version>8.0.17</version>
      </dependency>
      <dependency>
         <groupId>com.google.cloud.sql</groupId>
         <artifactId>mysql-socket-factory-connector-j-8</artifactId>
         <version>1.0.14</version>
      </dependency>
   </dependencies>

   <build>
      <plugins>
         <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-assembly-plugin</artifactId>
            <version>2.4.1</version>
            <configuration>
               <!-- get all project dependencies -->
               <descriptorRefs>
                  <descriptorRef>jar-with-dependencies</descriptorRef>
               </descriptorRefs>
            </configuration>
            <executions>
               <execution>
                  <id>make-assembly</id>
                  <!-- bind to the packaging phase -->
                  <phase>package</phase>
                  <goals>
                     <goal>single</goal>
                  </goals>
               </execution>
            </executions>
         </plugin>
      </plugins>
   </build>
</project>

Answer 2

It is a bit annoying to see "error: com.mysql.jdbc.Driver not found" and nothing more that can point to the root cause. Nevertheless, the error message in Data Fusion it is correct because the mysql Driver class is not included when the package is manually built . The issue is coming from this line:

<scope>provided</scope>

in the cloud-sql-jdbc-socket-factory/connector-j-8/pom.xml file which means that the target package won't include the mysql-connector-java jar file, the one that contains com.mysql.jdbc.Driver and others. The error can be resolved if you construct the jar connector without the aforementioned line.