How to INSERT INTO table having multiple column from dynamic query?

Question

Similar to this previous question for a one-to-one mapping I need a solution for multiple columns in source and destination.
Still working with Postgres 9.4.4, the query and schema are modified and are as below:

Let's say I have these two tables Table1 and Table2 :

Create table Table1(col1 int, col2 varchar(100), col3 varchar(100));
Create table Table2(col1 int, col2 varchar(100), col3 varchar(100));

There is another table Table3 storing a formula for migrating data from Table1 to Table2 :

CREATE TABLE Table3 (     
  tbl_src character varying(200),
  col_src character varying(500),
  tbl_des character varying(200),
  col_des character varying(100),
  condition character varying(500)
);

Insert into Table3(tbl_src, col_src, tbl_des, col_des, condition)
VALUES ('Table1','col1','Table2','col1', 'WHERE col1>=1')
     , ('Table1','col2','Table2','col2', NULL)
     , ('Table1','col3','Table2','col3', NULL);

How to compile this formula in a dynamic query and insert into the destination table?

Answer 1

The basic query to build the statement for multiple columns dynamically - ignoring the condition column:

SELECT format(
      'INSERT INTO %I (%s) SELECT %s FROM %I'
     , tbl_des
     , string_agg(quote_ident(col_des), ', ')
     , string_agg(quote_ident(col_src), ', ')
     , tbl_src) AS sql
FROM   table3
WHERE  tbl_des = 'Table2'
AND    tbl_src = 'Table1'
GROUP  BY tbl_des, tbl_src;

Result:

INSERT INTO "Table2" (col1, col2, col3)
SELECT CASE col1, col2, col3 FROM "Table1"

This assumes a single source and a single destination table. Or things get more complicated. I added WHERE conditions to make that clear. The notes on case-sensitivity I added to my previous answer still apply.

The above still ignores the condition . First of all, don't include the keyword WHERE in the condition column. That's just noise and not helpful:

INSERT INTO Table3(tbl_src, col_src, tbl_des, col_des, condition)
VALUES ('Table1','col1','Table2','col1', )  -- without WHERE!
     , ('Table1', ...

Warning

This approach is inherently unsafe. The condition holds expressions that need to be concatenated "as is", so you are completely open to SQL injection attacks . You need to make sure that untrusted users cannot write to table3 in any way to avoid this.

Building on this, and assuming each condition shall apply to its respective column only , we can solve it by wrapping the column in a CASE expression:

SELECT format(
      'INSERT INTO %I (%s) SELECT %s FROM %I'
     , tbl_des
     , string_agg(quote_ident(col_des), ', ')
     , string_agg(
         CASE WHEN condition IS NULL
            THEN quote_ident(col_src)
            ELSE format('CASE WHEN %s THEN %I END'
                 , condition, col_src)  -- condition is unsafe!
         END, ', ')
     , tbl_src) AS sql
FROM   table3
WHERE  tbl_des = 'Table2'
AND    tbl_src = 'Table1'
GROUP  BY tbl_des, tbl_src;

Produces a statement of the form:

INSERT INTO "Table2" (col1, col2, col3)
SELECT CASE WHEN col1>=1 THEN col1 END, col2, col3 FROM "Table1"

Or, like you added in a later comment, conditions can apply to whole rows . Logically, this is another grey area. The condition is stored with a particular column, but applies to the whole row ...

Be that as it may, you can add universal conditions in a WHERE clause.

SELECT format(
      'INSERT INTO %I (%s) SELECT %s FROM %I%s'
     , tbl_des
     , string_agg(quote_ident(col_des), ', ')
     , string_agg(quote_ident(col_src), ', ')
     , tbl_src
     , ' WHERE ' || string_agg(condition, ' AND ')) AS sql
FROM   table3
WHERE  tbl_des = 'Table2'
AND    tbl_src = 'Table1'
GROUP  BY tbl_des, tbl_src;

Conditions are ANDed, and the WHERE clause is only appended if there are any conditions - else the resulting NULL values swallows the added keyword in the expression ' WHERE ' || string_agg(condition, ' AND ')) ' WHERE ' || string_agg(condition, ' AND '))

Use this in a DO command or plpgsql function to execute dynamically like instructed in my previous answer:

• How to INSERT INTO table from dynamic query?

Basic plpgsql function:

CREATE OR REPLACE FUNCTION f_test()
  RETURNS void AS
$func$
BEGIN
   EXECUTE (
   SELECT format(
         'INSERT INTO %I (%s) SELECT %s FROM %I%s'
        , tbl_des
        , string_agg(quote_ident(col_des), ', ')
        , string_agg(quote_ident(col_src), ', ')
        , tbl_src
        , ' WHERE ' || string_agg(condition, ' AND ')) AS sql
   FROM   table3
   WHERE  tbl_des = 'Table2'
   AND    tbl_src = 'Table1'
   GROUP  BY tbl_des, tbl_src
   );
END
$func$ LANGUAGE plpgsql;

SQL Fiddle.