When using psql to copy from a csv file, a “” for a null column generates an error instead of a null

Question

I'm trying to use copy to copy a large csv file into a postgres table.

A certain integer column is primarily null. In the csv file, this column just has "".

Every column is quoted, which doesn't seem to be an issue for other columns.

I get this error when I try to copy it:

ERROR: invalid input syntax for integer: ""

I tried setting a NULL clause to '' and "" in my copy statement. '' does nothing, "" generates an error:

zero-length delimited identifier at or near """"

I tried using sed to change all "" to " ", but that still doesn't work even when I set the null clause to " ". I still get

ERROR: invalid input syntax for integer: " "

For now I am able to proceed by sed'ing the column to -1. I don't really care about this column much anyways. I'd be ok to just setting it to null, or ignoring it, but when I tried to take it out of the column definition section of the copy command, postgres yelled at me.

So my question comes down to this: how can I tell postgres to treat "" as a null value?

Thank you.

Answer 1

The typical way to indicate a missing value (null) in a.csv file is to just put nothing into that field. For instance, if you have three columns ( A , B and C ) and there is no value for B , the.csv file would contain "Col A value",,"Col C value" . "" is a string value, not a numeric value, so there's no way for it to be considered one.

Answer 2

This is what the force_null option is for:

Match the specified columns' values against the null string, even if it has been quoted

So assuming the name of the int column is "y":

\copy foo from foo.csv with (format csv, force_null (y));