从S3复制到Redshift无法识别换行符

Question

I am trying to run a COPY command from an S3 bucket to a Redshift PostgreSQL table, and I am getting the following error (in stl_load_errors):

err_code: 1207 

err_reason: Invalid digit, Value '2', Pos 3, Type: Short 

raw_field_value: 
2
2/28/15

The file has 2 lines:

2/28/15,Phone,Android,0,1,3,2,2

2/28/15,Phone,Android,0,4,1,2,2

The CREATE TABLE code is:

create table aggregate_table( date date, variable varchar(15),source varchar(15), prepaid smallint, direction smallint, total smallint, carrier smallint, carrier_group smallint)

It seems like the newline is not being recognized, and is trying to read the end of the first line and the beginning of the second line as one value. I have tried using delimiter ',' and escape , but nothing seems to work.

Thank you for your help!

Edit: Here's the COPY command (i've also tried it with escape at the end as well)

COPY aggregate_table FROM 's3://path_to_file.csv'  CREDENTIALS 'aws_access_key_id=XXXX;aws_secret_access_key=XXXXX'  CSV delimiter ','  DATEFORMAT AS 'MM/DD/YY'; 

Answer 1

You need to add DATEFORMAT AS 'MM/DD/YY' to your COPY command. Otherwise redshift can not parse date in first column correctly, as it expects YYYY-MM-DD .

See http://docs.aws.amazon.com/redshift/latest/dg/r_DATEFORMAT_and_TIMEFORMAT_strings.html for more details.

Answer 2

@quarterdome thanks for working through this with me! After you pointed out that it worked, I tried from the beginning to end again. It turns out that when I saved the file without a .csv extension, it worked! –