GetElementById always returns null, no matter what object/website i try

Question

Trying to automate actions on a website. I followed a tutorial and still no luck, no matter what website, tag or class I try it always gets null.

Saw a similar problem somewhere else where they suggested the website hadn't loaded so should add "while" section, which still didn't work for me.

webBrowser1.Navigate("https://www.simplesite.com/");
while (webBrowser1.ReadyState != WebBrowserReadyState.Complete)
{
     Application.DoEvents();
}
var myElement = webBrowser1.Document.GetElementById("_ctl0_Header2017_btnLogin");
myElement.InvokeMember("Click");

This is just a simple windows forms application. I've tried webBrowser1_Navigated and webBrowser1_DocumentCompletedmthods too.

Inspect element

Answer 1

The site is responding based on how it sees your browser's capabilities, which it determines based on the User Agent value passed in the request header. You need to set the User Agent string. The one the web browser control is sending is different than your Chrome/Firefox/IE sends.

I've added the Chrome User Agent string... you can find other User Agent strings here, under the Software section... https://developers.whatismybrowser.com/useragents/explore/

// call navigate and pass the latest Chrome User Agent string

webBrowser1.Navigate("https://www.simplesite.com/", null, null, 
    "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36\r\n");
while (webBrowser1.ReadyState != WebBrowserReadyState.Complete)
{
    Application.DoEvents();
    // let's just wait a few milliseconds but it would be better
    // if we used the DocumentCompleted event
    System.Threading.Thread.Sleep(100); 
 }
 var myElement = webBrowser1.Document.GetElementById("_ctl0_Header2017_btnLogin");
 myElement.InvokeMember("Click");