PSQL - Copy from csv file if column item does not exist

Question

I am trying to import a csv file into the postgres table where I can successfully do so using COPY FROM :

import.sql

\copy myTable FROM '..\CSV_OUTPUT.csv' DELIMITER ',' CSV HEADER;

But that query only adds rows if it is currently not in the database, otherwise it exits with an error. Key (id)=(#) already exists.

myTable

  id  |    alias    |    address            
------+-------------+---------------
  11  |   red_foo   |   10.1.1.11
  12  |  blue_foo   |   10.1.1.12

CSV_OUTPUT.csv

  id  |    alias    |    address            
------+-------------+---------------
  10  | black_foo   |   10.1.1.11
  12  |  blue_foo   |   10.1.1.12
  13  |  grey_foo   |   10.1.1.13
  14  |  pink_foo   |   10.1.1.14

My desired output is to insert the rows from the csv file into postgresql if address does not exist. myTable should contain grey_foo and pink_foo already but not black_foo since its address already exist.

What should be the right queries to use in order to achieve this? Your suggestions and ideas are highly appreciated.

Answer 1

Copy the data into a staging table first, and then update your main table ( myTable ) with only the rows with the keys that don't already exist. For example, assuming you have imported the data into a table named staging :

with nw as (
    select s.id, s.alias, s.address
    from staging as s
    left join mytable as m on m.address=s.address
    where m.address is null
    )
insert into mytable
    (id, alias, address)
select id, alias, address
from nw;

If you can upgrade to Postgres 9.5, you could instead use an INSERT command with the ON CONFLICT DO NOTHING clause.