Self-hosted agent (for Azure pipelines) using incorrect version of MSBuild

Question

When building a solution using Azure's self-hosted build agents, I am explicitly setting the MSBuild version to be used, both in the MSBuild task in the pipeline, but also as part of the build arguments.

However, when the build is executing, it is using a much older version of MSBuild but I'm not able to determine a cause. The Azure settings are so;

I have also verified that it is v12.0 that is running as I can see in the logs in diagnostic mode ;

39>GenerateTargetFrameworkMonikerAttribute:
   Skipping target "GenerateTargetFrameworkMonikerAttribute" because all 
   output files are up-to-date with respect to the input files.
   CoreCompile:
     C:\Program Files (x86)\MSBuild\12.0\bin\Csc.exe 
     /noconfig /nowarn:1701,1702 
     /nostdlib+ /platform:x86 
     /errorreport:prompt /warn:4 
     /highentropyva+ 
     /reference:"C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.8\mscorlib.dll" 
     /reference:"C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.8\System.Configuration.dll" 
     /reference:"C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.8\System.Core.dll" 
     /reference:"C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.8\System.dll" 
     /reference:C:\Users\<user>\Files\BuildAgents\ProdAgent1\_work\12\s\src\packages\Thinktecture.IdentityModel.Owin.ResourceAuthorization.1.1.0\lib\net45\Thinktecture.IdentityModel.Owin.ResourceAuthorization.dll 
     /debug:pdbonly /filealign:512 
     /optimize+ /out:obj\Release\Common.Security.dll 
     /subsystemversion:6.00 
     /target:library 
     /utf8output Authorization\CoreAuthorizationManager.cs 
                 Configuration\FrameworkMode.cs 
                 Configuration\IdentityConstants.cs                      
                 Configuration\LoggingConfig.cs                      
                 Configuration\SecurityConfig.cs                      
                 Configuration\Settings\ConfigFileLoggingSettingsProvider.cs                      
                 Configuration\Settings\ConfigFileSecuritySettingsProvider.cs                      
                 Configuration\Settings\ILoggingSettingsProvider.cs                      
                 Configuration\Settings\ISecuritySettingsProvider.cs                      
                 Configuration\Settings\LoggingSettings.cs                      
                 Configuration\Settings\SecuritySettings.cs                      
                 Configuration\Settings\SettingsHelper.cs                      
                 Configuration\TokenClientConfig.cs                      
                 Configuration\WebClientConfig.cs                      
                 Diagnostics\ILogger.cs                      
                 Diagnostics\Logger.cs                      
                 Diagnostics\LogHelper.cs                      
                 Diagnostics\LogHelper.Exceptions.cs                      
                 Extensions\ClaimsExtensions.cs                      
                 Extensions\ResourceAuthorizationContextExtensions.cs                      
                 Properties\AssemblyInfo.cs                      
                 SecurityFrameworkException.cs "C:\Users\{user}\AppData\Local\Temp\.NETFramework,Version=v4.8.AssemblyAttributes.cs"

I have also checked the user's and system environment variables to see if there was anything set and there is nothing I can find (in the code or environment), that explicitly references MSBuild version 12.0 . How do I force the agent to use 17.0 ?

The yaml for the MSBuild task is;

steps:
- task: MSBuild@1
  displayName: 'MSBuild solution src/{sln}.sln'
  inputs:
    solution: src/{sln}.sln
    platform: '$(BuildPlatform)'
    configuration: '$(BuildConfiguration)'
    msbuildArguments: '/verbosity:normal/p:VisualStudioVersion=17.0 /m:1 /p:GenerateSerializationAssemblies=Off /p:Platform="Auto"'
    clean: true
    createLogFile: true

UPDATE

I've applied several suggestions below in the comments and have gotten the following results;

1 What was tried

Installing a new build agent

1 Result

Using the latest version of the Azure agent, and installing it to the same azure pool, still results in the agent picking MSBuild V 12 , instead of the configured setting in the pipeline task.

---

2 What was tried

Changing the build task to a Visual Studio build.

2 Result

The build still picks MSBuild V 12 to do the build itself.

---

3 What was tried

Specifying the location of msbuild.exe , instead of specifying the version.

3 Result

This did pick the correct msbuild.exe at the location specified, but now the build breaks looking for a targets file... in the v 12 MSBuild folder. The error message is;

C:\Program Files (x86)\MSBuild\12.0\bin\Microsoft.Common.CurrentVersion.targets(67,3): 
Error MSB4019: The imported project "C:\Program Files (x86)\MSBuild\12.0\bin\Microsoft.NETFramework.props" was not found. 
Confirm that the expression in the Import declaration "Microsoft.NETFramework.props" is correct, and that the file exists on disk.

---

4 What was tried

Running a Repair on the Visual Studio version

4 Result

No effect. Still picking incorrect version of msbuild.exe

---

5 What was tried

Manually running the correct version of msbuild.exe against the solution the agent downloaded

5 Result

Build fails...but does actually build most of the projects. It fails on 9/~60 projects in the solution. The message for each of the failed projects looks like;

"C:\agent\_work\1\s\src\{sln}.sln" (default target) (1) ->
"C:\agent\_work\1\s\src\Tools\MakeTestSmartCard\MakeTestSmartCard.csproj" (default target) (133) ->
  C:\Program Files (x86)\MSBuild\12.0\bin\Microsoft.Common.CurrentVersion.targets(67,3): 
error MSB4019: The imported project "C:\Program Files (x86)\MSBuild\12.0\bin\Microsoft.NETFramework.props" was 
   not found. Confirm that the expression in the Import declaration "Microsoft.NETFramework.props" 
   is correct, and that the file exists on disk. 
[C:\agent\_work\1\s\src\Tools\MakeTestSmartCard\MakeTestSmartCard.csproj]

It almost works. But there's still something redirecting part of the build.

---

6 What was tried

Creating a basic test project and running msbuild against that.

6 Result

This works and completes successfully. It suggests that there may be something in the solution or solution's build configuration that is causing the build agent to select v12 of msbuild .

Answer 1

What versions of Visual Studio are all installed on this VM? A customer had similar issues and they reinstalled visual studio.

Can you check the version that is setup at the Agent Capabilities:

https://stackoverflow.com/a/52828794/8843952

Furthermore this looks at bit the same:

https://developercommunity.visualstudio.com/t/azure-pipelines-msbuild-vstest-cannot-find-msbuild/937549

update

Since you think VS2022 install has messed up your VM, these link could be usefull in the search for MSBuild:

• https://github.com/Microsoft/vswhere/wiki/Find-MSBuild
• https://gist.github.com/nicolasdao/b1623cb3220be3b6dc8fe04b2b2bfe32

Additional advice from wade Zhou msft

upgrade to vsbuild: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/vsbuild-v1?view=azure-pipelines

FYI just checked at the customer where we did the reinstall, for all yaml pipelines they're using vsbuild now. This might do the trick for you too.

Answer 2

In the end I wasn't able to determine the cause for the mis-used paths and ended up creating a new VM from a clean OS image, then installing VS2019 and VS2022.

While this ultimately worked, I did note that when the VM was configured to connect to the Azure agent pool, and the MSBuild task specified the version to use (17), the use of the command line switch /p:VisualStudioVersion=16.0 , caused it to overwrite msbuild s own settings.

In my case, I needed other assemblies to compile a .sqlproj . These were (correctly) stored in C:\Program Files\Microsoft Visual Studio\2022\Enterprise\MSBuild\Microsoft.Data.Tools.Schema.SqlTasks.targets .

This file then referenced the required assemblies in v17 of msbuild . However, the switch caused it to modify the settings stored in its configuration file and overwrite the \17\ part of the path to the assemblies and replace it with \16\ instead. This then broke the build as the resolved path didn't point to a valid location.

While not the original issue, it did demonstrate that there's definitely...odd...path manipulation going on within msbuild as it resolves them.