将数据从 Amazon redshift 卸载到 Amazon s3

Question

I am trying to use the following code to unload data into S3 bucket. Which works but after unloading it throws some error.

 Properties props = new Properties();
 props.setProperty("user", MasterUsername);
 props.setProperty("password", MasterUserPassword);
 conn = DriverManager.getConnection(dbURL, props);
 stmt = conn.createStatement(); 
 String sql;
 sql = "unload('select * from part where p_partkey in (select p_partkey from 
       part limit 10)') to"
       + " 's3://redshiftdump.****' "
       + " DELIMITER AS ','"
       + "ADDQUOTES " 
       + "NULL AS ''"
       + "credentials 'aws_access_key_id=****;aws_secret_access_key=***' "
       + "parallel off" + 
       ";"; 
 boolean i = stmt.execute(sql);
 stmt.close();
 conn.close();

The unloading works. It is creating a file in the bucket. But it is giving me some error

   java.sql.SQLException: 
      dataengine.impl.DSISimpleRowCountResult cannot be cast to 
      com.amazon.dsi.dataengine.interfaces.IResultSet
   at 
   com.amazon.redshift.core.jdbc42.PGJDBC42Statement.createResultSet(Unknown 
   Source)
   at com.amazon.jdbc.common.SStatement.executeQuery(Unknown Source)

what is this error and how to avoid it? Is there any way to dump the table in CSV format. Right now it is dumping the file in FILE format.

Answer 1

You say the UNLOAD works but you receive this error, that suggests to me that you are connecting successfully but there is an problem in the way your code interacts with the JDBC driver when the query completes.

We provide an example that may be helpful in our documentation on the page "Connect to Your Cluster Programmatically"

Regarding the output file format, you will get whatever is specified in your UNLOAD SQL but the filename will have a suffix (for example "000" or "6411_part_00") to indicate which part of the UNLOAD it is.

Answer 2

use executeUpdate .

  def runQuery(sql: String)  = {
    Class.forName("com.amazon.redshift.jdbc.Driver")
    val connection = DriverManager.getConnection(url, username, password)
    var statement: Statement = null
    try {
      statement = connection.createStatement()
      statement.setQueryTimeout(redshiftTimeoutInSeconds)
      val result = statement.executeUpdate(sql)
      logger.info(s"statement response code : ${result}")
    } catch {
      case e: Exception => {
        logger.error(s"statement.isCloseOnCompletion :${e.getMessage} ::: ${e.printStackTrace()}")
        throw new IngestionException(e.getMessage)
      }
    }
    finally {
      if(statement != null ) statement.close()
      connection.close()
    }
  }