TFS 2017 - Scheduled build fails some unit tests, Gated checkin works

Question

I have a Winforms application (MVP architecture) using .NET 4.6.2 and C#7 enabled.

In TFS 2017 we have 2 build definitions for the app: 1. a Gated checkin - which works all the time. 2 A Scheduled build that runs at 11PM Mon-Fri. - which doesn't work.

The build definitions are pretty much identical (nuget restore, VS Build with 2017, run tests...) and build the same solution. The only difference is that the scheduled overnight build additionally runs code analysis and code coverage is enabled. Platform & Config is Any CPU | Release.

The scheduled overnight build fails on 8 tests. Some tests invoke async methods in the SUTs but others are pretty vanilla, so I don't see any pattern.

The test projects depend on the following nuget packages:

• MSTest.TestAdapter 1.1.18
• MSTest.TestFramework 1.1.18
• Moq 4.2.1507.118

The only thing I see that is different is that the Gated checkin is queued under the developer's name, whereas the scheduled evening build is kicked off by Microsoft.TeamFoundation.System

TL:DR version: How can I diagnose why the tests won't run at night, yet work just fine with the Gated checkin?

Answer 1

Pleas try below items to narrow down the issue:

• Check if the definitions are using the same build agent, if not just try with the same build agent.
• Deploy a new agent for the scheduled build definition.
• Keep all the same for both build definitions except the Gated Check-in option. That means disable code analysis and code coverage for the scheduled build definition.
• Also enable the Gated Check-in for the scheduled definition.

Then check that again.

According to this thread , there is an issue for the scheduled build on earlier version of TFS 2017. So, if you are using the version earlier than Update1 RC2, then you can try upgrading to the latest version, then try it again.

For version check, please refer to this link : https://blogs.msdn.microsoft.com/tfssetup/2013/11/21/what-version-of-team-foundation-server-do-i-have/