Running Selenium Tests in Chrome (Headless mode) on a VSTS Hosted Agent

Question

Is it possible to run Selenium C# tests on a hosted agent within a Visual Studio Team Services build/release using the Chrome driver in headless mode?

I've seen contradictory information indicating this is and isn't possible currently. This user voice idea seems to suggest Chrome is pre-installed on the hosted agents:

https://visualstudio.uservoice.com/forums/330519-visual-studio-team-services/suggestions/19387174-install-headless-chrome-on-hosted-build-servers

Current Setup

As part of the setup of my tests (in code), I'm indicating that I want the Chrome driver to run in headless mode and they do so when run within a local instance of VS.

Example:

var options = new ChromeOptions();
options.AddArgument("headless");
options.AddArgument("disable-gpu");

var driver = new ChromeDriver(options);

When the tests are run as part of the release definition within VSTS, the following error is thrown:

2018-02-20T13:21:10.7954002Z Error Message:
2018-02-20T13:21:10.7954156Z  unknown error: cannot find Chrome binary
2018-02-20T13:21:10.7954340Z   (Driver info: chromedriver=2.35.528161 (5b82f2d2aae0ca24b877009200ced9065a772e73),platform=Windows NT 10.0.14393 x86_64)
2018-02-20T13:21:10.7954487Z Stack Trace:
2018-02-20T13:21:10.7954620Z System.InvalidOperationException: unknown error: cannot find Chrome binary
2018-02-20T13:21:10.7955947Z   (Driver info: chromedriver=2.35.528161 (5b82f2d2aae0ca24b877009200ced9065a772e73),platform=Windows NT 10.0.14393 x86_64)
2018-02-20T13:21:10.7956136Z    at OpenQA.Selenium.Remote.RemoteWebDriver.UnpackAndThrowOnError(Response errorResponse)
2018-02-20T13:21:10.7956387Z    at OpenQA.Selenium.Remote.RemoteWebDriver.Execute(String driverCommandToExecute, Dictionary`2 parameters)
2018-02-20T13:21:10.7956557Z    at OpenQA.Selenium.Remote.RemoteWebDriver.StartSession(ICapabilities desiredCapabilities)
2018-02-20T13:21:10.7956729Z    at OpenQA.Selenium.Remote.RemoteWebDriver..ctor(ICommandExecutor commandExecutor, ICapabilities desiredCapabilities)
2018-02-20T13:21:10.7956927Z    at OpenQA.Selenium.Chrome.ChromeDriver..ctor(ChromeDriverService service, ChromeOptions options, TimeSpan commandTimeout)
2018-02-20T13:21:10.7957106Z    at OpenQA.Selenium.Chrome.ChromeDriver..ctor(String chromeDriverDirectory, ChromeOptions options)

I've ensured the chromedriver.exe has been copied over to the bin directory and is included in the build artifacts that are being used by the release process.

Alternatives

I know you can setup a private agent to run these tests on, but I wanted to avoid doing this if possible and make use of the hosted agents within VSTS.

I've also managed to successfully run the tests using the PhantomJS driver within VSTS, but again we want to be able to test exclusively against the Chrome browser if possible.

UPDATE

As per the accepted answer below (by Bernard Vander Beken), I managed to resolve this by inserting an additional release task to install Chrome silenty and unattended via a PowerShell script. This task (obviously) needs to be excuted prior to the test run so that the browser can then be used in headless mode within the Selenium tests.

Answer 1

It should be possible, see https://github.com/Microsoft/vsts-agent/issues/1378

The hosted agent is running as Admin, so you should be able to install chrome as part of your build as long as chrome has unattended installation mode.

Answer 2

I was having Same Issue. This issue is mostly because of Version Compatibility, I have tried some combinations and make it work. Earlier I used latest version of ChromeDriver ie 76.*.*.* but this is not supported on any of Hosted Agent (As on date of writing this answer) I was using Win 1803 . I have later changed my ChromeDriver Version to 72.*.*.* and Hosted Agent to VS2017-Win2016 and it is working fine now. It also works with Windows2019 .

So conclusion is DevOps takes some time to make support available for latest release. Try out variety of combinations based on your project environment.

Answer 3

In the more recent Hosted VS Agents, eg Hosted VS2017 , web browsers and even Selenium drivers are already installed. You can follow documentation for hosted agents at the Azure Piplelines Image Generation Github Repo . For example, here is the reference to Chrome being present on the VS2017 agent .

Click the Agent Phase title at the top of your pipeline. Then under Agent pool select Hosted VS2017 rather than the vanilla Hosted pool. ( Windows 2019 with VS 2019 has also recently become available).