是否可以运行Visual Studio Team Services Build中的端到端测试？

Question

I have a set of mocha tests that launch Spectron to do end-to-end testing on an Electron application.

When I try to run the tests through npm run , they just time out and fail. It seems that the application isn't launching. There is a Run Functional Tests build step template in VSTS, but that's for an actual test DLL. Is there something similar for Javascript end-to-end testing?

Answer 1

The Hosted Agent is running as the service, however launch Spectron to do test with Electron app need to interact with window and the test is running on build agent machine. So you need to setup a on premise build agent to do that test (works fine for me with this test sample. Build steps: npm install, npm install test)

1. Log on to the machine using the account for which you've prepared permissions as explained above.
2. Go to the Agent pools control panel tab
3. Click Download agent.
4. Click Windows.
5. Click the Download button.
6. Run PowerShell as Administrator.
7. Run the commands under Create the agent.
8. Run .\\config cmd

More information to setup on premise build agent, you can refer to this article.

Answer 2

Since the tests need to interact with the desktop they are not a good fit for compile time where you dont usually have a running instance of your application. Stick to pure unit tests at build time. A better fit would be to use Release Management to orchastrate those tests.

That said, you can easily deploy your application (build or release) and the. Use the Functional Testing task to initiate your UI interactive tasks. You need a machine with the Test Agent deployed, but there is a task for that too.

Since Functional (end to end) tests interact with the desktop you can only run one at once. This is fine if you only take a few minutes, but as you get more functional teats this can take way longer. The Functional Testing task allows you to split the test execution across multiple agents so that you can run the tests in parallel.

Since your alternative is to configure your own Build Agent in interactive mode you need a server anyway. You would be more future proofing to use this model from the start as you can scale.

If a Visual Studio Test Adapter is not available for your framework then you can create your own to let Visual Studio, and additionally the Functional Testing plugin, detect and execute any test framework. For Javascript you will find http://mortenhoustonludvigsen.github.io/JsTestAdapter/CreatingATestAdapter/ has some awesome examples of how to create a Jasmin Test Adapter as an example.