Error 405 Method not Allowed on WebRequest

Question

I am trying to grab the page code from the below page. It gives me a 405 error. If I try to get the page code from the home page it works fine but from this specific page i get Method not allowed, thoughts?

        WebRequest request = WebRequest.Create("https://www.realtor.com/realestateandhomes-search/California/counties");
        request.UseDefaultCredentials = true;
        request.Credentials = CredentialCache.DefaultCredentials;
        HttpWebResponse response = (HttpWebResponse)request.GetResponse();
        Stream dataStream = response.GetResponseStream();
        StreamReader reader = new StreamReader(dataStream);
        string responseFromServer = reader.ReadToEnd();

        Console.WriteLine(responseFromServer);

Answer 1

The site thinks you are a bot.

Details:

I tried it with HttpClient (recommended: doesn't throw an exception upon receiving a non-200 response code), and inspected the response HTML. Here is the important snipit:

      <p>
        As you were browsing, something about your browser made us think you might be a bot. There are a few reasons this might happen, including:
      </p>
      <ul>
        <li>You're a power user moving through this website with super-human speed</li>
        <li>You've disabled JavaScript and/or cookies in your web browser</li>
        <li>A third-party browser plugin is preventing JavaScript from running. Additional information is available in this
          <a title='Third party browser plugins that block javascript' href='http://ds.tl/help-third-party-plugins' target='_blank'>
            support article
          </a>.
        </li>
      </ul>

If you want the full response, try running this:

async void LogResponse()
{
    using System.Net.Http.HttpClient client = new System.Net.Http.HttpClient();
    var response = await client.GetAsync("https://www.realtor.com/realestateandhomes-search/California/counties");
    Console.WriteLine(await response.Content.ReadAsStringAsync());
}

Side complaint against realtor.com, 405 ( The method specified in the Request-Line is not allowed ) is a rather poor response code for this; a 403 ( The server understood the request, but is refusing to fulfill it. ) seems better suited.