How do I copy data from a CSV dump into a PSQL database with a different structure?

Question

I'm currently trying to copy a SQLite3 database onto a PostgreSQL database with a DIFFERENT structure. How do I do that?

What I did so far:

1. I dumped the SQLite database and created CSV files with headers for all tables.
2. I then created a new PSQL database with a NEW table structure that suits my needs.
3. I also created a PSQL database with the old SQLite table structure and imported the data from the mentioned CSV files so I have access to both databases, the one with the old schema and the new one, just in case I need it.

The problem:

I don't know how to import/copy the data if the table and column names differ from the original ones. Eg the SQLite database and the CSV have the following structure for the table „keyword“:

ID, CreatedBy, CreatedByUserId, CreatedOn, ModifiedBy, ModifiedByUserId, ModifiedOn, Name, Notes, Protected, ReservedData

The new PSQL database has the following structure for the differently named table „keywords“:

KeywordsID, CreatedOn, Keyword, ModifiedOn, Notes

Obviously a simple

COPY keywords(
            keywordsid, createdon, keyword, notes)
FROM '/home/user/Desktop/bib_csv/keywords.csv' DELIMITER ',' CSV;

will not work.

Background:

The original database was created by a bibliographical program called Citavi. It started out as a Microsoft Access database and was later ported to SQLite3 by the program itself, apparently the database now contains some errors that prevent most programs to assist with the export to PSQL, which means I have to do it manually.

I would like to have online access to the database, but for a number of reasons I want it to run on PostgreSQL instead of SQLite.

Answer 1

You can't import columns selectively with copy . If you don't want to use a different tool, the easiest way of doing this is to create a staging table that has the original structure, then COPY into that table and then copy only some columns using an insert statement:

create table keywords_old
(
  ID ..., 
  CreatedBy ..., 
  CreatedByUserId ..., 
  CreatedOn, ...
);

COPY keywords_old
FROM '/home/user/Desktop/bib_csv/keywords.csv' DELIMITER ',' CSV;

insert into keywords (keywordsid, createdon, keyword, notes)
select id, createdon, name, notes
from keywords_old;

Another option is to export only the selected columns from the old database:

COPY keywords (id, createdon, name, notes)
  TO '/home/user/Desktop/bib_csv/keywords.csv' DELIMITER ',' CSV;

Then you can import that file into the new database.

---

You can make all that a lot easier, if you import the old data into the same database but into a different schema (eg old_data ) instead of public you can just copy the data using plain insert statements:

insert into public.keywords (keywordsid, createdon, keyword, notes)
select id, createdon, name, notes
from old_data.keywords;