Dynamically copy data from one table to another

Question

The script is great when I have one table:

INSERT INTO new_Table1 (col1, col2, col3)
SELECT col1, col2, col3
FROM old_Table1

Now I have 100 tables need to copy to another 100 tables, new tables all have some data that can't delete, so I can't create new 100 tables, but handling with 100 tables is too much work, can I write a dynamic script for it? SQL Server is 2019.

Answer 1

Executing a dynamic statement is an option, but the important question is how do you generate each table name. One possible approach is to generate this statement using system catalog views:

DECLARE @stm nvarchar(max) = N''

SELECT @stm = @stm + 
    N'INSERT INTO ' +
    QUOTENAME(CONCAT(N'new_', STUFF([name], 1, 4, N''))) +
    N' (col1, col2, col3) SELECT col1, col2, col3 FROM ' + 
    QUOTENAME([name]) +
    '; '
FROM sys.tables
WHERE [name] LIKE 'old_%'

PRINT @stm
EXEC sp_executesql @stm

As an additional option, if you want to include the schema name:

SELECT @stm = @stm + 
    N'INSERT INTO ' +
    CONCAT(QUOTENAME(sch.[name]), N'.', QUOTENAME(CONCAT(N'new_', STUFF(tab.[name], 1, 4, N'')))) +
    N' (col1, col2, col3) SELECT col1, col2, col3 FROM ' + 
    CONCAT(QUOTENAME(sch.[name]), N'.', QUOTENAME(tab.[name])) +
    '; '
FROM sys.tables tab
JOIN sys.schemas sch ON tab.schema_id = sch.schema_id
WHERE tab.[name] LIKE 'old_%'
PRINT @stm
EXEC sp_executesql @stm

Finally, to get the column names dynamically:

DECLARE @stm nvarchar(max) = N''

SELECT STRING_AGG(t.Stmt, ' ')
FROM (
   SELECT CONCAT(
      N'INSERT INTO ' +
      CONCAT(QUOTENAME(sch.[name]), N'.', QUOTENAME(CONCAT(N'new_', STUFF(tab.[name], 1, 4, N'')))) +
      N' (',
      STRING_AGG(QUOTENAME(col.[name]), ','),
      N') SELECT '+
      STRING_AGG(QUOTENAME(col.[name]), ','),
      N' FROM ' + 
      CONCAT(QUOTENAME(sch.[name]), N'.', QUOTENAME(tab.[name])) +
      ';'
   ) AS Stmt
   FROM sys.columns col
   JOIN sys.tables tab ON col.object_id = tab.object_id
   JOIN sys.schemas sch ON tab.schema_id = sch.schema_id
   WHERE tab.[name] LIKE 'old_x%'
   GROUP BY sch.[name], tab.[name]
) t

PRINT @stm
EXEC sp_executesql @stm

Answer 2

I am creating two old tables and corresponding two new tables. Old tables are holding the data, new tables does not hold data. I am generating script for loading them from old table to new table.

CREATE TABLE old_customs(id int, val int)
CREATE TABLE new_customs(id int, val int)

CREATE TABLE old_feature(id int, val int)
CREATE TABLE new_feature(id int, val int)

INSERT INTO old_customs
VALUES(1,1),(2,2),(3,3);
INSERT INTO old_feature
VALUES(1,1),(2,2),(3,3);

DECLARE @tableList TABLE(oldTableName SYSNAME, newTableName SYSNAME)

INSERT @tableList
SELECT name, REPLACE(name,'old','new') from sys.tables WHERE name like 'old%'

SELECT CONCAT('INSERT INTO ', newTableName, ' SELECT * FROM ', oldTableName,';',CHAR(13),CHAR(10),'GO')
FROM @tableList

Generated script

INSERT INTO new_customs SELECT * FROM old_customs;
GO
INSERT INTO new_feature SELECT * FROM old_feature;
GO

Answer 3

My script allows for variable field lists and different data types. It contains multiple assumptions, but they are all fullfilled here:

• The old and the new table have the same amount of fields.
• The field order of the old table is the same as the field order of the new table. So OldField1 corresponds with NewField1 .
• The corresponding fields have compatible types.

The script:

-- declare temp table to store table name mapping
declare @tableMapping table
(
    TableFrom   nvarchar(100),
    TableTo     nvarchar(100)
);

-- define mapped tables: use editor of preference to generate similar lines (regex, Excel, ...)
insert into @tableMapping (TableFrom, TableTo) values
('Old_Table1', 'New_Table1'),
('Old_Table2', 'New_Table2'),
('Old_Table3', 'New_Table3');

-- declare temp table to hold all statements (to select in bulk at the end)
declare @statements table
(
    Stmt    nvarchar(max)
);

-- declare helper variables for looping and field lists
declare @tableFrom  nvarchar(100);
declare @tableTo    nvarchar(100);
declare @fieldsFrom nvarchar(2000);
declare @fieldsTo   nvarchar(2000);

-- loop the mapping table
declare mapCursor cursor for
select tm.TableFrom, tm.TableTo  
from @tableMapping tm
order by tm.TableFrom;

open mapCursor;

fetch next from mapCursor
into @tableFrom, @tableTo  

while @@FETCH_STATUS = 0
begin
    -- build field list FROM
    set @fieldsFrom = '';

    select @fieldsFrom = @fieldsFrom + c.name + ', '
    from sys.columns c
    where c.object_id = object_id(@tableFrom);

    -- build field list TO
    set @fieldsTo = '';

    select @fieldsTo = @fieldsTo + c.name + ', '
    from sys.columns c
    where c.object_id = object_id(@tableTo);

    -- build copy statement
    insert into @statements (Stmt)
    values (N'insert into ' + @tableTo
         + N' (' + substring(@fieldsTo, 0, len(@fieldsTo))
         + N') select ' + substring(@fieldsFrom, 0, len(@fieldsFrom))
         + N' from ' + @tableFrom + N';');

    fetch next from mapCursor
    into @tableFrom, @tableTo 
end

close mapCursor;
deallocate mapCursor;

-- print all generated statements
select * from @statements;

My 6 sample tables

Old_Table1 [Col1, Col2, Col3] ==> New_Table1 [Fld1, Fld2, Fld3] --> field names can change
Old_Table2 [Col1, Col2]       ==> New_Table2 [Col1, Col2]       --> number of fields does not matter
Old_Table3 [Col1, Col2, Col3] ==> New_Table3 [Col1, Col2, Col3] --> the exact question

Output for my 6 sample tables

insert into New_Table1 (Fld1, Fld2, Fld3) select Col1, Col2, Col3 from Old_Table1;
insert into New_Table2 (Col1, Col2) select Col1, Col2 from Old_Table2;
insert into New_Table3 (Col1, Col2, Col3) select Col1, Col2, Col3 from Old_Table3;