SSIS / SQL Server XML

Question

I'm working creating a XML output from SQL Server 2008R2. Below the structure of the xml I want. (I will try to be clear as much as possible, but if you need more information, please let me know) Issue I'm having is with the node "RecordId". This field has to be a running sequence throughout the XML, irrespective of under which node it is. ie, each occurrence of the node "RecordId" under any category will be 1 value more than the immediate previous one, even if it is under a different category.

The main constraint I have is, I will have to strictly use T-SQL, SSIS only (can have script task with VB or C#).

<Root>
  <includedFileHeader>
    <GenDatetime>2017-01-13T11:53:36</GenDatetime>
    <OFtype>PD</OFtype>
    <issuerID>ABCDE</issuerID>
    **<RecordId>1</RecordId>**
  </includedFileHeader>
  <includedIssuerResult>
    <issuerId>ABCDE</issuerId>
    **<RecordId>2</RecordId>**
    <includedPlanResult>
      <planId>2</planId>
      <insPlanId>123456789</insPlanId>
      **<RecordId>4</RecordId>**
      <ClassStatusType>
        <Code>A</Code>
      </ClassStatusType>
      <includedDetails>
        <DetailId>48</DetailId>
        <DetailClmId>A3456H567</DetailClmId>
        **<RecordId>4</RecordId>**
      </includedDetails>
      <includedDetails>
        <DetailId>74</DetailId>
        <DetailClmId>163364170257204</DetailClmId>
        **<RecordId>5</RecordId>**
      </includedDetails>
    </includedPlanResult>
    <includedPlanResult>
      <planId>3</planId>
      <insPlanId>343546337</insPlanId>
      **<RecordId>6</RecordId>**
      <ClassStatusType>
        <Code>A</Code>
      </ClassStatusType>
      <includedDetails>
        <DetailId>55</DetailId>
        <DetailClmId>A78947J780</DetailClmId>
        **<RecordId>7</RecordId>**
      </includedDetails>
      <includedDetails>
        <DetailId>44</DetailId>
        <DetailClmId>146545165A54</DetailClmId>
        **<RecordId>8</RecordId>**
      </includedDetails>
  </includedIssuerResult>
</Root>

I couldn't achieve even close in T-sql, but tried to use some VB/C# code in script task, using XMLReader, Streamreader/writer, nothing worked. Any help in this is much appreciated.

Answer 1

In general I'd avoid to deal with XML on string level. In this case this might be a workaround...

DECLARE @xml XML=
N'<Root>
  <includedFileHeader>
    <GenDatetime>2017-01-13T11:53:36</GenDatetime>
    <OFtype>PD</OFtype>
    <issuerID>ABCDE</issuerID>
    <RecordId>99</RecordId>
  </includedFileHeader>
  <includedIssuerResult>
    <issuerId>ABCDE</issuerId>
    <RecordId>33</RecordId>
    <includedPlanResult>
      <planId>2</planId>
      <insPlanId>123456789</insPlanId>
      <RecordId>22</RecordId>
      <ClassStatusType>
        <Code>A</Code>
      </ClassStatusType>
      <includedDetails>
        <DetailId>48</DetailId>
        <DetailClmId>A3456H567</DetailClmId>
        <RecordId>66</RecordId>
      </includedDetails>
      <includedDetails>
        <DetailId>74</DetailId>
        <DetailClmId>163364170257204</DetailClmId>
        <RecordId>11</RecordId>
      </includedDetails>
    </includedPlanResult>
    <includedPlanResult>
      <planId>3</planId>
      <insPlanId>343546337</insPlanId>
      <RecordId>6</RecordId>
      <ClassStatusType>
        <Code>A</Code>
      </ClassStatusType>
      <includedDetails>
        <DetailId>55</DetailId>
        <DetailClmId>A78947J780</DetailClmId>
        <RecordId>7</RecordId>
      </includedDetails>
      <includedDetails>
        <DetailId>44</DetailId>
        <DetailClmId>146545165A54</DetailClmId>
        <RecordId>8</RecordId>
      </includedDetails>
    </includedPlanResult>
  </includedIssuerResult>
</Root>';

--The query

WITH Splitted AS
(
    SELECT CAST('<x>' + REPLACE((SELECT CAST(@xml AS NVARCHAR(MAX)) FOR XML PATH('')),'&lt;RecordId&gt;','</x><x>') + '</x>' AS XML) AS Part
)
,Parted AS
(
    SELECT p.value(N'(./text())[1]',N'nvarchar(max)') AS line
         ,ROW_NUMBER() OVER(ORDER BY (SELECT NULL))-1 AS nr 
    FROM Splitted
    CROSS APPLY Part.nodes(N'/x') AS A(p)
)
SELECT CAST(
        (SELECT line FROM Parted WHERE nr=0)
     +  (
        SELECT  '<RecordId>' + CAST(nr AS NVARCHAR(10)) + '</RecordId>' + SUBSTRING(line,CHARINDEX('<',line,5),10000)
        FROM Parted 
        WHERE nr>0
        ORDER BY nr     
        FOR XML PATH(''),TYPE 
        ).value('(./text())[1]',N'nvarchar(max)') AS XML);

Short explanation

The first CTE Splitted uses an XML based on-the-fly string-split approach. The problem here is, that characters like < or > would let this break. That's why I use SELECT ... FOR XML PATH('') before the actual split. This will escape all forbidden characters implicitly. Next step is to split the string at <RecordId> , which is now <RecordId>

The second CTE Parted uses .nodes() to get a list derived table and ROW_NUMBER to get a running number.

The final SELECT re-concatenates the parted lines inserting the numbered <RecordId> in the right places.

Answer 2

Thank you everyone for your suggestion and help here. I managed to come up with a solution (similar to Laughing Vergil's thought) (although not that neat, but able to get what I want at the sametime not very slow).

This is what I did: I created a scalar UDF, that will take 2 inputs 1.XML data, 2.Name of the tag where I want the sequence (in this case it is "RecordID", just to reuse the same if in future needed for a different tag). Also, I defaulted the " <RecordID/> " field to 0 when I generate the XML, this will be searched and replaced within.

What the function does: 1. Convert the input XML to VARCHAR(MAX) 2. Find the first occurrence of the " <RecordID/> " and Stuff it with #1 as sequence 3. Keep looping through the XML until it finds the last match and for each occurrence Stuff it with the appropriate loop number (@i)

CREATE FUNCTION dbo.UDF_UPDATE_RECORDID_XML (
  --XML Data
  @xmld VARCHAR(MAX),
  --Tag/Node name for the RecordID
  --ex: if <RecordID>0</RecordID>, then pass 'RecordID'
  @rec_tag VARCHAR(100)
)
RETURNS XML
AS
BEGIN


  --o/p param
  DECLARE @op_xmld VARCHAR(MAX)

  --processing params
  DECLARE @i INT
  DECLARE @j INT
  DECLARE @prev_i INT
  --search pattern
  DECLARE @pat VARCHAR(100) = '<' + @rec_tag + '>0</' + @rec_tag + '>'

  SELECT @op_xmld = @xmld

  --Find first occurance of the pattern in the XML
  SELECT @i = PATINDEX('%' + @pat + '%', @op_xmld)
        , @j = 1

  SELECT @prev_i = @i

  --Loop until there is no match for the pattern
  WHILE (@i > 0)
  BEGIN
        --Replace the identified substring with the sequence number in between
        SET @op_xmld = STUFF(@op_xmld, @i, LEN(@pat), '<' + @rec_tag + '>' + CAST(@j AS VARCHAR(20)) + '</' + @rec_tag + '>')

        --sequence number
        SET @j = @j + 1

        --store the current index value of the pattern
        --so the next search can happen after this point
        SET @prev_i = @i

        --Search down for the pattern from the previous index value
        SELECT @i = CHARINDEX(@pat, @op_xmld, @prev_i + LEN(@pat))
  END

      RETURN CAST(@op_xmld AS XML)
END