想要将Redshift表数据卸载到S3，并在文件名上附加日期

Question

UNLOAD ('SELECT * FROM cn.test') TO 's3://bucket/archive/stage/test-#{format(@scheduledStartTime,'YYYY-MM-dd-HH')}.csv' credentials 'aws_access_key_id=###;aws_secret_access_key=###' DELIMITER '|' ADDQUOTES PARALLEL OFF ALLOWOVERWRITE

I tried the above command to unload to S3. Getting error : Amazon Invalid operation: syntax error at or near " YYYY "; 1 statement failed.

If i give escape character near to date, the file name is going like test-# {format(@scheduledStartTime,'YYYY-MM-dd-HH')}.csv instead of actual sysdate .

Could anyone can help me

Answer 1

A known issue. I had similar requirements. As a resolution, I wrote a shell script with the unload statement and when it executed successfully, I fired an mv command to rename the file on the s3 bucket.

script:

psql -U$user -h hostname -p port -c "unload (...) to 's3://bucket-name' credentials 'aws_access_key_id; aws_secret_access_key' [switch1][switch2]..."

    if [ "$?" != "0" ]
        then
                echo "\nS3 Error: Error in unloading CSV to S3"
                echo "\nS3 Error: Error in unloading CSV to S3" >> $log_file

        else    aws s3 mv s3://bucket-name/"$file_name".csv000 s3://bucket-name/"$file_name$from_date".csv
                echo "$file_name$from_date successfully copied to s3" >> $status_file
                return 0
        fi