How to Parse text file using SSIS Script Component with Complex and Varied Data

Question

I have a text file that holds data based on a series of tags. Tag A represents a column with CRLF delimiter, Tag B is a separate column with a semi-colon delimiter, and Tag C contains values delimited by under scores and CRLF delimiters. I've created a C# Script component with output columns for each tag.

Sample Data for each Tag:

Tag A varchar(10) A123456789 
TagB varchar (50) cdw5w3464353
Tag C varchar(max) ***;User_____Date______Value;User123_____6/3/15_____3366cde;

Do I use RegEx for identification of each tag value? Or is there another way to handle this scenario? Note that the tags repeat for each entry to be loaded into the SQL Server destination. Also, note that only some of the tags are needed, extraneous text values in the records can be disregarded.

Answer 1

I would try to keep the Script component as simple as possible - just splitting the input rows into separate text files, perhaps adding Row Numbers or similar to keep track of sets of rows. I would use Strings.Split to chop each Tag C row into separate columns, eg all the Date values go in column #2, with a single heading row showing "Date" in column #2.

Then I would use the SSIS Flat File Source component (in a subsequent Data Flow Task) to load the data from the separate text files. That Data Flow can ignore any unwanted columns.