What are the ways to join foreign key for fact table in ETL package?

Question

I am a SSIS developer, this is the question i ran into when I design the package.

what are the ways people use to grab foreign key for fact table form dimension table? Assuming your staging DB and Data Warehouse are not in the same Server so simple T-SQL join across DB is not an option.

The way I am doing now is loading the data from StagingDB into a temp table in Data Warehouse and they do the T-SQL join there to grab foreign key.

In SSIS package we have Lookup component, but I don't think it's an option if you have more than 10 key to join

Wondering what are the option I have here? thanks!

Answer 1

I have found that using staging tables and SQL to look up foreign keys, the way you are doing now, is more performant than the lookup component in my experience.

The staging table doesn't have to be a temp table. You can designate a permanent table to be the "staging" table, and then use sql to move to the destination table. Just be sure to truncate the table before each ETL run.