I get HttpWebResponse with unclear characters when use a class in web application

Question

I created a class in Windows Form application that uses HttpWebResponse .

This class gets the response from some page before login and after login and I analyze this pages.

Everything is good working at all time.

BUT

I used this class in a web application and uploaded project files in server.

I get good response from all page EXCEPT login page . Login page response contains unclear characters like this:

�      �Z�N#G�v��CM�'��X؎2�LvV�e�H{��݅����t�
(ڋa���z.��2�͒H��膇 
�+�9U�v�n��$pW��S�|��s�)=z����ߗ����0*T��~JuF5y�Ǟ���D��

and it happens often , that means I get sometimes normal response.

For example when extract my zip file again in server, it may work fine.

I am confused, I can't tell regular rule.

URL = "http://www.nonsense-website.com/login";
var request2 = (HttpWebRequest)WebRequest.Create(URL);

string username1 = jandoe;
string password1 = pass1234;

string postData = "hidlogin=1&username=" + username1 + "&password=" + password1;

var data = Encoding.UTF8.GetBytes(postData);

request2.Method = "POST";
request2.ContentType = "application/x-www-form-urlencoded";
request2.ContentLength = data.Length;
request2.UserAgent = "Mozilla/5.0 (Windows NT 10.0; rv:55.0) Gecko/20100101 Firefox/55.0";
request2.Headers.Add("Cookie:" + cookie);
request2.Headers.Add("Accept-Language: en-US,en;q=0.5");
request2.Headers.Add("Accept-Encoding: gzip, deflate");
request2.KeepAlive = true;
request2.Headers.Add("Upgrade-Insecure-Requests: 1");

using (var stream = request2.GetRequestStream())
{
    stream.Write(data, 0, data.Length);
}

try
{
    using (var response2 = (HttpWebResponse)request2.GetResponse())
    {
        responseString = new StreamReader(response2.GetResponseStream(),Encoding.UTF8).ReadToEnd();
        AnalyzeResponse(responseString); // other function that analyze response data
    }
}
catch (Exception ex)
{
    WriteData("Exception in Login" + Environment.NewLine + ex.Message); // write in file
}

and my exception:

Exception in Login
Error in Parse the HTML
Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index

Answer 1

You're accepting gzip and deflate compressed responses (via Accept-Encoding ), but you aren't checking whether the response is compressed. Turn on automatic decompression to have this done for you:

request2.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;