Populate SQL from xml file using dataset, c#

Question

I have a big xml file, from which I get many tables:

DataSet l_DataSet = new DataSet();
l_DataSet.ReadXml(@"D:\file.xml");

tables that I have in dataset:

Channels: id_table, id_channel, name* * *.
Links: id_table, id_channel, url.
Countries: id_table, id_channel, country.

Now I have to merge them in some way to insert into 1 table:

SQLtable: id_channel, code, description (may be null), url, country.

*The main problem is: **Code and description is a "name" value from Channels table, which looks like:
id_table id_channel code/description
1 1 code1
2 1 description1
3 2 code2
4 2 description2
5 3 code3
6 4 code4
7 4 description4
sometimes there is description missing.

I don't know how can I merge tables in the dataset in c#? Some directions?

Thank you!

Answer 1

If I knew more about your table structure and XML document I can try and give you a better example

Depending on how many rows you are inserting I would add the data in chunks to help not lock up the database. Typically I pass 25,000 chunks at a time. If your rows are more than 25000 then for loop through it only doing 25000 at a time.

On the SQL Side create a procedure:

SET ANSI_NULLS ON
GO

SET QUOTED_IDENTIFIER ON
GO

CREATE Procedure [dbo].[prI_BulkAddData](@Data Xml)
As

 -- Add Handle to XML Data
 Declare @xmlDataHandle Int
 Exec sp_xml_preparedocument @xmlDataHandle Output, @Data

 Insert Into TableName(Column1, Column2, Column3)
 Select ElementName1, ElementName2, ElementName3 From OpenXml(@xmlDataHandle, '/Root/Channel', 2)
 With(
    [ElementName1] int,
    [ElementName2] varchar(50,
    [ElementName3] datetime
     )

      -- Remove Handle to Free-Up Memory
 Exec sp_xml_removedocument @xmlDataHandle

GO

On your visual studio side Call the Procedure:

EXEC prI_BulkAddData Data = 'PUT XML HERE'

This is just an example obvious you will need to write some C# code to read the XML file and to Execute a Stored Procedure (I typically use Linq to SQL as I am lazy and it works good and dont want to write a bunch of data code)

In the Stored Proc make sure you adjust it to how your XML doc is wrote.

Here is an example I did once to parse large XML files:

  for (int t = 0; t < data.Data.Count(); t = t + 25000)
                            {
                                var xEle = new XElement("DataCollections",
                                        from emp in data.Data.Skip(t).Take(25000)
                                        select new XElement("DataCollection",
                                                   new XElement("AssetID", emp.AssetID),
                                                   new XElement("DataPointID", emp.DataPointID),
                                                   new XElement("SourceTag", emp.SourceTag),
                                                   new XElement("TimeStep", emp.TimeStep),
                                                   new XElement("RetInterval", emp.RetInterval),
                                                   new XElement("DatapointDate", emp.DatapointDate.ToString()),
                                                   new XElement("DataPointValue", emp.DataPointValue),
                                                   new XElement("DataFlagID", emp.DataFlagID),
                                                   new XElement("DateDataGrabbed", emp.DateDataGrabbed.ToString()),
                                                   new XElement("DateAddedToDB", emp.DateAddedToDB.ToString())
                                        ));

                                _DataCollectorManager.Services.dataProcessService.BulkAddPIData(xEle);

                            }