How to automatically import XML file into various new tables using SSIS

Question

I have a very large XML file (with an xsd file) which has many different objects that need to be imported into SQL Server tables. When I mean objects, I mean the highest level XML wrapping tag eg Products, Orders, Locations etc

My current method using SSIS has been to:

1. Create a Data Flow Task
2. Set up a new XML source in the data flow task
3. Drag a new OLE DB Destination into the task
4. Link the XML output to the input of the OLE DB Destination
5. Open the OLE DB Destination task and ask it to create a new table for the data to go into

I have to repeat steps 3-5 for all the objects in the XML file which could run into the hundreds. Theres no way that I can manually do this.

Is there anyway to get SSIS to just create new tables for all the different objects in SQL server and import the data into those? So it would automatically create dbo.Products, dbo.Locations, dbo.Customers and put the correct XML data into those tables.

I can't see any other feasible way of doing this.

Answer 1

Is there anyway to get SSIS to just create new tables for all the different objects in SQL server and import the data into those?

No :(

There's really two problems, here. You have not gotten to the second one, yet, which is ssis is probably going to choke reading a very large xml file. The XML source component loads the entire file into memory when it reads it.

There are a couple of alternatives that I can think of: - use XSLT transforms - roll your own sax parser and use a script component source

For the XSLT method, you would transform each object into a flatfile ie parse just your customer data into a csv format and then add data flows to read in each flat file. The drawbacks are that ssis uses an earlier version of XSLT which also loads the whole file into memory, rather than streaming it. However, I have seen this perform very well on files that were 0.5 GB in size. Also, XLST can be challenging to learn if you are not already familiar, but it is very powerful in getting data into a relational form.

The sax parser method would allow you to stream the file and pull out the parts that you want into a relational form. In a script component, you could direct different objects to different outputs.