是否有获取一些真实世界ETL示例的资源？

Question

I'm fully convinced that a significant part of the work I'm doing falls into this domain of ETL, but I didn't even know the term existed before 3 months ago. I've found SSIS to be a bit of a mismatch for my skillset, ie my instincts are that writing C# code in a well thought out way will give me the result I need (also my employer doesn't own it). I started looking at WF because if seemed logical, but I came back to the original conclusion that I really need to understand the fundamentals of the problem domain, and when I do that it will make the most sense to leverage my experience and code the solution in .net/c# (I'm a one man team and that doesn't seem to be changing). So far I have a sort of hodge-podge of syncher utilities, and it was the difficultly that began arising in managing them all that led to seek out this knowledge.

QUESTION 1 is: is there a resource for me to get some examples of how it's all put together for things like:

• extracting from REST services with usage limits --> loading to databases for purposes of (as close to) real time (as possible) synchronization
• extracting from in-house 3rd party apps like QuickBooks --> loading to databases
• monitoring for changes in database and updating external systems in carefully tracked batches (ie the same information that was extracted is changed by an LOB app and then needs to be pushed back)

QUESTION 2 is: I've yet to grasp where the T part will come into play. Thus far I've been pulling the information that represents logical entities in one system and pushing them into another.

Answer 1

I don't have any examples of the exact scenarios your looking at, but if you want to learn more about ETL itself, you can try taking a look at the articles on Ayende's site . He has an extremely easy to use framework for ETL processes called Rhino ETL . And a video showing how to use it .

As for where the T part comes in to play, the T stands for Transform. This is the step in the process where you can (but do not necessarily have to) change the shape of the data. After Extracting from one datasource, you can add or remove fields, aggregate information, break objects up in to tables, map tables into objects, etc. This part is the transform step. You then proceed to Load the data in to the new data storage or system.

Hope that helps some.

Answer 2

http://en.wikipedia.org/wiki/Extract,_transform,_load

Will help too.