How to bypass comma , double quote using Lumenworks

Question

I am fetching data from a database in csv format.ie http://iapp250.dev.sx.com:5011/a.csv?select[>date]from employee

But some columns in the table contains comma and double quotes and after reading the csv it becomes comma separated string.As a result I get index out of bound exception when I deserialize it.

I decided to use Lumenworks csv reader after reading some of the posts.I proceeded in below way but still could not fix it.Please find the below code snippet.

List<List<string>> LineFields = new List<List<string>>();
using(var reader = new StreamReader(siteminderresponse.getResponseStream())
{
 Char quotingCharacter = '\0'; 
        Char escapeCharacter = quotingCharacter;
        Char delimiter = '|';
        using (var csv = new CsvReader(reader, true, delimiter, quotingCharacter, escapeCharacter, '\0', ValueTrimmingOptions.All))
        {
            csv.DefaultParseErrorAction = ParseErrorAction.ThrowException;
            //csv.ParseError += csv_ParseError;  
            csv.SkipEmptyLines = true;

            while (csv.ReadNextRecord())
            {
                List<string> fields = new List<string>(csv.FieldCount);
                for (int i = 0; i < csv.FieldCount; i++)
                {
                    try
                    {
                        string field = csv[i];
                        fields.Add(field.Trim('"'));
                    } catch (MalformedCsvException ex)
                    {
                                                throw;
                    }
                }
 LineFields Add(fields);
            }
        }

}

End result is I get comma separated fields like

1,16:00:01,BUY,-2,***ROBERTS, K***.

Please see the ROBERTS, K which was a single column value and now is comma separated due to which my serialization fails.

Answer 1

This appears to be a problem with the way you're formatting the results of your query. If you want "ROBERTS, K" to be read as a single field by a CSV reader, you need put quotes around "ROBERTS, K" in your input to the CSV reader. LumenWorks is only doing what it's supposed to.

Similarly, if you want literal double-quotes in your parsed fields, you need to escape them with another double-quote. So, to properly express this as a single field:

Roberts, K is 6" taller than I am

...you'd need to pass this into the CSV parser:

"Roberts, K is 6"" taller than I am"