xml encoding special characters

Question

I am trying to send a XML rest message and it has special characters in the fields. When I send it when decoded it is putting ? in where the special characters should be.

    ServiceUtil.Log(xmlRequest.OuterXml);
            TransactionFactory.Response(xmlRequest, ServiceUtil.SERVICE, ExistingReceipt.poNumber);
            ServiceUtil.Log("Written to transaction factory.");
            objHttpWebRequest = (HttpWebRequest)WebRequest.Create(BrokerDetails.endpoint);
            ServiceUtil.Log(string.Format("Endpoint {0}", BrokerDetails.endpoint));
            byte[] bytes = Encoding.ASCII.GetBytes(xmlRequest.InnerXml);
            objHttpWebRequest.Method = "POST";
            objHttpWebRequest.Headers["Authorization"] = string.Concat("Basic ", Convert.ToBase64String(Encoding.Default.GetBytes(string.Format("{0}:{1}", BrokerDetails.username, BrokerDetails.password))));
            ServiceUtil.Log(string.Format("User {0}, password {1}", BrokerDetails.username, BrokerDetails.password));
            objHttpWebRequest.ContentLength = (long)((int)bytes.Length);
            objHttpWebRequest.ContentType = "application/xml; encoding='utf-8'";
            objRequestStream = objHttpWebRequest.GetRequestStream();
            objRequestStream.Write(bytes, 0, (int)bytes.Length);
            objRequestStream.Close();
            XmlDocument xmlDoc = new XmlDocument();
            objHttpWebResponse = (HttpWebResponse)objHttpWebRequest.GetResponse();
            ServiceUtil.Log(string.Format("Got status {0}", objHttpWebResponse.StatusCode.ToString()));
            if (objHttpWebResponse.StatusCode == HttpStatusCode.OK)
            {
                objXMLReader = new XmlTextReader(objHttpWebResponse.GetResponseStream());
                xmlDoc.Load(objXMLReader);
                if (xmlDoc.FirstChild.NodeType == XmlNodeType.XmlDeclaration)
                {
                    xmlDoc = BaseImpl.StripDeclerations(xmlDoc);
                }
                xmlDoc.LoadXml(xmlDoc.DocumentElement.OuterXml);
                objXMLReader.Close();
                TransactionFactory.Response(xmlDoc, ServiceUtil.SERVICE, ExistingReceipt.poNumber);
                Response = BaseImpl.BuildResponse<GoodsReceivedResponse> (xmlDoc);
                ServiceUtil.Log(string.Format("Response {0}", Response.Result));
                Transmited = true;
            }
            else if (Tries <= 4)
            {
                Response = null;
                Tries += 1;
            }
            else
            {
                ErrorMessages.AddAnError(new Error(string.Format(ServiceUtil.NULL_RESPONSE, ExistingReceipt.poNumber)));
                Response = new GoodsReceivedResponse(ErrorMessages.ErrorMessages);
                Transmited = true;
                ServiceUtil.Log("No response");
            }

This is the code that is encoding and sending the request out.

    <stockReceivedRequest>
     <stockReceived>
<distributorCode>GEO</distributorCode>
<dateReceived>2016-06-10T00:00:00</dateReceived>
<timeReceived>2016-06-10T01:01:01</timeReceived>
<courier>ABC</courier>
<poNumber>1122</poNumber>
<stockDetailReceived>
  <stockDetail>
    <productCode>G0-4MK</productCode>
    <altProductCode />
    <productName>Gift set ü</productName>
    <quantity>10</quantity>
  </stockDetail>
</stockDetailReceived>

This is the XML we are trying to send and this is what is being received.

    <?xml version="1.0"?>
    <stockReceivedRequest>
<stockReceived>
    <distributorCode>GEO</distributorCode>
    <dateReceived>2016-06-10T00:00:00</dateReceived>
    <timeReceived>2016-06-10T01:01:01</timeReceived>
    <courier>ABC</courier>
    <poNumber>1119</poNumber>
    <stockDetailReceived>
        <stockDetail>
            <productCode>G0-4MK</productCode>
            <altProductCode />
            <productName>Gift set ?</productName>
            <quantity>10</quantity>
        </stockDetail>
    </stockDetailReceived>
</stockReceived>

Hope someone is able to help, if you need anything else just ask.

Regards

Aidan

Answer 1

Your XML has non-ASCII characters in it, but you are encoding it using ASCII. Any character that's not part of the ASCII character set will just be encoded as ? , which is what you're seeing.

Change this:

Encoding.ASCII.GetBytes(xmlRequest.InnerXml)

To this:

Encoding.UTF8.GetBytes(xmlRequest.InnerXml)

Or, better still, just save the XmlDocument directly to the stream. The default encoding for this is UTF-8 :

xmlRequest.Save(objRequestStream);