How do I run a coded UI Test using Jenkins?

Question

For the past few days I have been cracking my head trying to figure out how I can run a Coded UI Test directly from a Jenkins Job. Since i am quite new to jenkins. I plan on running the jenkins job using the test controller and agent both installed in the same machine as the local VS project repository where my project is stored.

My goal is to get a Jenkins Job to run a coded ui test which interacts directly with my desktop.

Specification used for the test:

  1. VS Enterprise 2015
  2. Agents for MS 2013
  3. Jenkins 1.69

What I have Done So far is Below (this is the batch command that i run from Jenkins, which i copied from How to run Coded UI test file from Visual Studio command prompt using batch file? ) which works Successfully when run through CMD :

@echo off
:: Running tests without VS Enterprise is possible if you install the Test Agent package: https://msdn.microsoft.com/en-us/library/dd648127.aspx

set test_runner="C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe"
set test_dlls="C:\Location\Compiled\Tests\Project.CodedUI.Test.dll"

:: If tests is set then only these comma separate test cases are run
:: set tests="Test1,Test2"
set tests=""

if %tests% == "" (
     %test_runner% %test_dlls% > CodedUITestResults.txt
) else (
     %test_runner% %test_dlls% /tests:%tests%
)
pause

I am using my local repository (Not TFS or GIT since that is not the requirement) on my windows machine which is when running the script executes the coded ui test successfully.

However, when it comes to replicating the steps in jenkins I get this error (even after configuring the test controller using VS in a .settings file):/

Error calling Initialization method for test class CodedUITest: Microsoft.VisualStudio.TestTools.UITest.Extension.UITestException: To run tests that interact with the desktop, you must set up the test agent to run as an interactive process. For more information, see "How to: Set Up Your Test Agent to Run Tests That Interact with the Desktop" ( http://go.microsoft.com/fwlink/?LinkId=159018 ) /

My Questions are:

• Why does my tests keep giving me this error despite the fact that my test agent has been set as interactive?
• Also, how would I get it to run from Jenkins, in order for it to interact with desktop directly as an interactive process?

Answer 1

MS test controllers and agents are quite buggy and don't have proper documentation. This is why my company developed a free and open-source alternative. You can find it here- http://meissarunner.com There is a link to documentation where it will be quite easy for you to setup. Since, it uses .NET core you need to specify an additional parameter mentioning the .net Framework version you use --nativeRunnerArguments="--framework=net462" you can find a complete list of the frameworks here- https://docs.microsoft.com/en-us/dotnet/standard/frameworks If you have any problems drop me a line.

Start Meissa in server mode:

meissa.exe initServer

Then start an agent:

meissa.exe testAgent --testAgentTag="APIAgent" --testServerUrl="http://IPServerMachine:5000"

In Jenkins job start the runner:

meissa.exe runner --resultsFilePath="pathToResults\result.trx" --outputFilesLocation="pathToBuildedFiles" 
--agentTag="API" --testTechnology="MSTestCore" 
--testLibraryPath="pathToBuildedFiles\SampleTestProj.dll"

Answer 2

This is a little late but to make the agent interact with the desktop all you need to do is on the setup of the agent check the checkbox for run as I interactive process. You shouldn't need master and agent if it's all on the same box either. I think you can change the jenkins service to also interact with the desktop. Hopefully this helps somebody. Agents and master works but I'll definitely check out meissarunner and see if it's better.