SSIS Staging Table to Normalized form

Question

I could be down the wrong path with this. However, here goes. I am trying to take multiple excel sheets and load them into SQL Server using SSIS.

Excel sheet:

RQ|Descr|PartNum|Manufacturer|...

I am loading this into a staging table with a couple of derived columns:

RQ|Descr|PartNum|Manufacturer|Origin|DateTime|...

This is no big deal, I am able to do this easily. However, the problem is how to get the data from the staging table to the correct table and ensuring FK constraints are followed. See below for an illustration.

My goal is to take RQ|Descr|PartNum|Manufacturer|Origin|DateTime|... and populate multiple tables

[t1] id|RQ|Descr|Origin|DateTime

[t2] id|t1_id|PartNum|Manufacturer

[t3] id|t1_id|...

I have tried MERGE however I am unsure how to keep the FK relationship.

MERGE INTO spin_item AS targ

USING ssis_stage AS src ON 1=0 -- always generates "not matched by target"

WHEN NOT MATCHED BY TARGET THEN
    -- INSERT into spin_item:
    INSERT (description, reqqty, price, origin, datetime, exclude, status, siteid, production, repairable)      
    VALUES (src.description, src.rq, src.price, src.origin, GETDATE(), 0, 'N', '', 0, 0)

    -- INSERT into spin_part: 
    OUTPUT inserted.ID, src.manufacturer, src.partnum 
    INTO spin_part (ID, src.manufacturer, src.partnum);

I have looked into this SSIS : Using multicast to enter data into 2 RELATED destinations but this is for a one-to-many relationship. So, I am not sure how to populate my t1 table and use the id to populate t2, t3 from the staging table.

EDIT: Below, seems to be a working solution. However, I am not sure that it is a good solution.

BEGIN
SET IDENTITY_INSERT dbo.spin_item ON

--Insert into spin_item
MERGE INTO spin_item AS targ
USING ssis_stage AS src ON 1=0

WHEN NOT MATCHED BY TARGET THEN
    INSERT (id, description, reqqty, price, origin, datetime, exclude, status, siteid, production, repairable)
    VALUES (src.id, src.description, src.rq, src.price, src.origin, GETDATE(), 0, 'N', '', 0, 0);
SET IDENTITY_INSERT dbo.spin_item OFF

--Insert into spin_part
MERGE INTO spin_part AS targ
USING ssis_stage AS src ON 1=0

WHEN NOT MATCHED BY TARGET AND src.partnum IS NOT NULL THEN
    INSERT (itemid_id, manufacturer, partnum, catalognum, [primary])
    VALUES (src.id, src.manufacturer, src.partnum, src.partnum, 1);

--Insert into spin_stock
MERGE INTO spin_stock AS targ
USING ssis_stage AS src ON 1=0

WHEN NOT MATCHED BY TARGET AND src.stock IS NOT NULL THEN
    INSERT (itemid_id, stocknum)
    VALUES (src.id, src.stock);  


--Insert into spin_collaboration        
MERGE INTO spin_collaboration AS targ
USING ssis_stage AS src ON 1=0

WHEN NOT MATCHED BY TARGET AND src.notes IS NOT NULL THEN
    INSERT (itemid_id, comment, datetime)
    VALUES (src.id, src.notes, GETDATE());

DELETE FROM ssis_stage WHERE id > 0 --Instead of Truncate since auto_increment will reset.

END

Answer 1

You can create an ID column on your staging table, based off your target tables that is then used as the FK in each table insert:

declare @source table (ID int, a int, b int, c int);
insert into @source values
 (null,1,1,1)
,(null,1,1,2)
,(null,1,2,2)
,(null,5,3,2)
,(null,7,1,2)
,(null,2,1,2)

declare @target1 table (ID int, a int);
insert into @target1 values
 (1,5)
,(2,6)
,(3,99);

declare @target2 table (ID int, b int, c int);
insert into @target2 values
 (1,3,2)
,(2,9,7)
,(3,57,3);

update s
set ID = ss.IDNew
from @source s
    inner join (
                select row_number() over (order by a,b,c) + (select max(ID) from @target1) as IDNew
                        ,a
                        ,b
                        ,c
                from @source
                ) ss
        on(s.a = ss.a
            and s.b = ss.b
            and s.c = ss.c
            );

select * from @target1;
select * from @source;

insert into @target1
select ID
        ,a
from @source;

insert into @target2
select ID
        ,b
        ,c
from @source;

select * from @target1;
select * from @target2;