Eliminate special characters from CSV with C#

Question

I am getting data from a CSV file through my Web Api with this code

private List<Item> items = new List<Item>();

        public ItemRepository()
        {
            string filename = HttpRuntime.AppDomainAppPath + "App_Data\\items.csv";

            var lines = File.ReadAllLines(filename).Skip(1).ToList();

            for (int i = 0; i < lines.Count; i++)
            {
                var line = lines[i];

                var columns = line.Split('$');

                //get rid of newline characters in the middle of data lines
                while (columns.Length < 9)
                {
                    i += 1;
                    line = line.Replace("\n", " ") + lines[i];
                    columns = line.Split('$');
                }

                //Remove Starting and Trailing open quotes from fields
                columns = columns.Select(c => { if (string.IsNullOrEmpty(c) == false) { return c.Substring(1, c.Length - 2); } return string.Empty; }).ToArray();


                var temp = columns[5].Split('|', '>');
                items.Add(new Item()
                {
                    Id = int.Parse(columns[0]),
                    Name = temp[0],
                    Description = columns[2],

                    Photo = columns[7]



                });
            }
        }

But the CSV file returned data with special characters instead of an apostrophe.

For example in the CSV file the are values such as There ’ s which should be "There's" or "John ’ s" which should be "John's". This ’ is there instead of an apostrophe.

How do I get rid of this to just show my apostrophe. This kind of data is being returned in Name = temp[0], Description = columns[2],

Answer 1

You can use the HttpUtility.HtmlDecode to convert the characters. Here's an example:

var withEncodedChars = "For example in the CSV file the are values such as There’s which should be There's or John’s which should be John's. This ’ is there instead of an apostrophe.";

Console.WriteLine(HttpUtility.HtmlDecode(withEncodedChars));

If you run this in a console app it outputs:

For example in the CSV file the are values such as There's which should be There's or John's which should be John's. This ' is there instead of an apostrophe.