Insert Unicode data from xml string to Datatable

Question

I want to save unicode data into database from xml string by using this code:

XmlDocument xmlDoc = new XmlDocument();
xmlDoc.LoadXml(xmlText);

using (XmlNodeReader xmlReader = new XmlNodeReader(xmlDoc))
{
        DataTable dt = new DataTable();
        dt.TableName = "sms";
        dt.Columns.Add("rowID");
        dt.Columns.Add("origAddr");
        dt.Columns.Add("time");
        dt.Columns.Add("message");
        dt.ReadXml(xmlReader);
        return dt;
}

but when I save datatable into database my unicode character appear with question mark (???????)

My database collation is correct and other unicode character are stored correctly.

Answer 1

I'll start things off with an educated guess.

Your database, or your table, uses a character set that is not full Unicode. The characters which are getting stored as question marks are characters which are outside the database or table character set. The characters which are getting stored correctly happen to be within the database or table character set.

Alternatively, you have your XMLDocument() or DataTable() objects are converting the characters they read into a character set which is less than full Unicode.

Give the extra information requested by the comments, and I'll see if I can improve this answer.

Answer 2

Usually this happens when you source text is not stored as Unicode. For example, if you read your xml data from a text file, and the text file is stored as Ansi (using codepage), or it is stored as Unicode file without BOM (Byte Order Mark, or signature), when you read your text file, non-ASCII characters may not be read correctly.
To solve this, open your source xml file in a text editor (for example Notepad++ ) and change your encoding to Unicode or UTF-8, and then save the file.
You can also open the file in Notepad, and save the file as Unicode (File/Save As -> Encoding: Unicode or UTF-8). Make sure that when you open your file in notepad, the characters are displayed correctly.

Answer 3

use XmlTextReader for read the xml and verify if error persist

XmlTextReader stream = new XmlTextReader(_pathXml);
     while (stream.Read())
     {
         //TODO save each element
     }