Loading Fact tables from SCD1 and SCD2 Dimension in SSIS

Question

I am finding it difficult to understand how you get the history data from a fact table join to a Dimension that has Type2 and Type1 for historic records that have changed. Currently I have a Surrogate Key and Business Key in the Dim. The Fact Table has the Surrogate Key the Fact table and I am using SSIS Lookup Component currently to bring back the row that has the CurrentFlag set to Yes.

However I am joining on the Business Key in the Lookup and returning the Surrogate. Which I know is the main reason I can't get history, however if I Join on the Business Key as I am currently doing and return the Business Key also, SSIS component will only bring back just one row, regardless of how many versions of history you have against that Business Key.

What I want to know or have been told is to use lookups to populate fact tables, however this doesn't seem to really give me the history as it will only return one row regardless. So I Just want to know how to return historic date between a fact and a dimension in SSIS.

Thank you

Answer 1

There's a few caveats when it comes to historical dimensions. Your end users will need to know what it is you are presenting, and understand the differences.

For example, consider the following scenario:

Customer A is located in Las Vegas in January 2017. They place an order for Product 123, which at that time costs $125.

Now, it's August. In the meantime, the Customer moved to Washington DC in May, and Product 123 was updated in July to cost $145.

Your end users will need to inform you what they want to see. In case you are not tracking history whatsoever, and simply truncate and load everything on a daily basis, your order report would show the following:

Customer A, located in Washington DC placed an order for $145 in January.

If you implement proper history tracking, and implemented logic to identify the start- and end-date of a row in a dimension, you would join the fact table to the dimension using the natural key as well as the proper date interval. This should return you a single value for every dimension row in the fact table. IF it returns more, you have overlapping dates.

Can you show us the logic where you receive only a single value from the lookup, even though you have more records?