两种不同数据库模式之间的数据映射

Question

I have to design a system with an sql database whose work is to get data from different databases which may be in another database such as mysql or oracle etc. Then the system will map the attributes of that database with my database schema and store them.. Example reference link: https://msdn.microsoft.com/en-us/library/aa728893(v=vs.71).aspx Since I am new, I can't attach images which is why I am providing links.

All my searches end up with getting the mapping tools but what actually I want is how to create that tool myself. I am not a professional but a little push will be enough for me and highly appreciated. Thanks in advance.

Answer 1

Please, ignore for this task, any form of programming, Do not do it. Especially ignore Entity framework.

I would recommend you to take one of the many ETL tools out there, and design the work with it. I can recommend you Talend . It is very easy to learn and in time, even a developer starts to work with it. The best part about it that it can connect to webservices, enterprise solutions and probably any imaginable Database out there.

What you do is to design Jobs that run on its own in parallel, and then you export the jobs as standalone JARs. A chroneJob, or a scheduled service completes the process to have it executed periodically

Answer 2

This, as I said in the comment, sounds to me a job for an Integration Services package.

If you would like to use Microsoft SQL Server Integration Services you should first have Microsoft SQL Server Data Tools installed on your development machine.

Afterwards you start by creating a new Integration Service project inside Visual Studio. Then you can add an ODBC Connection Manager to manage your different databases input of data. After that you can add a different transformation container objects in your package to transform the data as you need it. At the end you need to specify the output of all those elements into your database where you want to store the information that you collect from other sources.

You can also create a different package for every source database you have so that the tasks can be separated. Unfortunately a complete tutorial is very long for me to post here but you can check out the tutorial on Microsoft web site . An other example here .

As as warning you should be really careful with data types because if you don't match/convert them correctly the package will fail with not so obvious errors.

If you choose the .tt (T4 Template) solution in which you create the application then you should start by connecting to the sources database and loop through tables definitions to get the columns and then store them as a xml file. The matching you will have to do it inside the text template file so that you have the matching already done when the table is read from the data source.

Here is an example that should get you started. Note that in the example the output file will be a .cs file not .xml but you can configure that very easily with this T4 directive <#@ output extension=".xml" #> .