用户代理问题呈现不可读的标记html源数据

Question

Opening the following URL code inspection in Chrome yeilds the following message:

Refused to set unsafe header "User-Agent"

And the code behind the site looks like this:

<div dir="auto" class="rn-13yce4e rn-fnigne rn-ndvcnb rn-gxnn5r rn-deolkf rn-cme181 rn-1471scf rn-14xgk7a rn-1b43r93 rn-o11vmf rn-ebii48 rn-vw2c0b rn-t9a87b rn-1mnahxq rn-61z16t rn-p1pxzi rn-11wrixw rn-wk8lta rn-9aemit rn-1mdbw0j rn-gy4na3 rn-bauka4 rn-q42fyq rn-qvutc0">24 Count</div>

Opening the same link in Internet Explorer seems to render the tags properly, for example: gbcvs-c-addToCart__inner can be found.

Selenium is used to read the page, and it used to work with Chrome. Is there a way to set the user agent knowing the program will run in Linux?

I tried the following with no luck..

options = webdriver.ChromeOptions()
options.add_argument('--user-agent="Mozilla/5.0 (Windows NT 6.3; Trident/7.0; rv:11.0) like Gecko"')

I would like to keep using the chrome driver.

Answer 1

There is no generic way to do it, but for chrome you can use a plugin like this one to fake your user agent header. When using chrome from Selenium you can then load that extension and use it to change that header for all requests.