how to publish automation test result in MTM while using TFS 2017?

Question

I am setting up Test agents as part of build definition and now I want to run automation test case using TCM.exe, How can I pass the Test Environment name as an argument?

In TFS 2013 I did something like this-

start-Process -FilePath "C:\\Program Files (x86)\\Microsoft Visual Studio 12.0\\Common7\\IDE\\tcm.exe" -ArgumentList " run /create /title:Nightly Run /planid:1334 /suiteid:123 /configid:123456 /settingsname:Test_xyz_TestSettings /testenvironment:xyzFuntional /collection: http://xyz:8080/tfs/ppp /teamproject:aa /include"

because in 2013 we were setting test controller and test agent as part of lab management and we knew the name of the test environment.

My requirement is to publish automation result in MTM ,any other way is really appreciated .Please note i can not use Test Hub for Test result publishing.

Thanks

Answer 1

Based on your screenshot, you are using the Run functional Test but not TCM.exe.

VStest, MStest and TCM.exe are different tools (Refer to this thread for more info.).

The tests in your scenario supposed to be used to associate test runs with builds and actually using vstest. MTM is displaying test results that are associated with test plans/suites which are somewhat independent of builds.

You can try to use tcm.exe to get what you're after. It's a command line interface to running tests, similar to mstest, but it runs the tests through the infrastructure you set up in MTM. Here's the documentation page: http://msdn.microsoft.com/en-us/library/dd465192.aspx

However, you can also try to follow the steps mentioned in below link to publish the VStest or MStest test results to MTM :

Uploading automated test results to MTM made "easy"

Use TCM to upload the resulting TRX file

Note There is problem with VSTS (formerly VSO) that causes these automated tests to stay in a state of “In Progress”*

Sample TCM command line:

 TCM run /publish /suiteid:ID# /configid:CONFIG# /resultowner:"USER" /resultsfile:"C:\\DLL\\testResults.trx" /collection:"TFS_COLLECTION_URL" /teamproject:"PROJECT_NAME" 

---

UPDATE:

The infrastructure (eg Test Environment you mentioned here ) is set in MTM, If you run the test with tcm.exe, generally it will pick up the parameters automatically. But if run test with VSTest or MStest the parameters may need to be defined with Set up environments to run continuous test tasks with your build tasks ...

Whatever, based on your description I think below articles should helps:

• Continuous Delivery with VSO : Executing Automated Web Tests with Microsoft Test Manager
• Continuous Delivery with TFS : Running Automated Web Tests with MTM