Why does this UTF-16 HTTP response end up as UTF-8 when in the resulting Stream?

Question

I have an issue where a service is returning me a HTTP Header:

Content-Type: application/json; charset=utf-16

When this is serialised by C# this ends up in a UTF-8 stream, which obviously breaks. It seems that utf-16 is a valid encoding in IANA spec . So why is this code not working?

System.Net.Http.HttpClient httpClient ...;
using (var response = await httpClient.SendAsync(request, HttpCompletionOption.ResponseHeadersRead))
{
   //response.Content.Headers.ContentType.CharSet = "utf-16"
   using (var responseContentStream = await response.Content.ReadAsStreamAsync())
   {
       using (var streamReader = new StreamReader(stream))
       {
          //streamReader.CurrentEncoding.BodyName returns utf-8 here?! 
       }
   }
} 

so initially the response seems fine but then once it gets as far as the streamReader it seems to of reverted back to utf-8. Why?

Answer 1

You can specify the encoding the StreamReader should use in the constructor. In your case it should look like this:

using (var streamReader = new StreamReader(stream, Encoding.Unicode, true))
{
  // The reader should read the Stream with UTF-16 here
}