msbuild and visual studio project configuration problem

Question

I have a solution containing a lot of projects and installer projects. One project uses a third party package. The package comes with a native DLL and a .net wrapper DLL. In order for the code to work, the .net wrapper DLL needs to find the native DLL in runtime. But the code never directly refers to the native DLL in compile time (the code talks to .net wrapper DLL in compile time).

Now I have to choose proper way to deploy the native DLL, during compile time on a programmer's machine and during installing time on a user machine.

Basically I have two options, either to put the native DLL to Windows System folder or to put the native DLL in the local folder containing the exe file.

To put the native DLL to system folder, I need a post-build script to xcopy the file to the directory after building the solution. I also need to create a System Folder output in installer project for installer to work on a client machine. I don't know whether copying files to system folder is a good idea or not. For this solution, every time someone (else in the big team) creates a new installer, he or she has to remember to create System Folder output and add the native DLL under the configuration otherwise his or her installer will not install a workable piece of software on a user machine.

To put the native DLL to local folder, I have the following two ways: 1. Use a post-build script. This solution, I have to find out every executable project in my big solution that has a reference to the project using the native DLL and link the post-build script to every such executable project. In the future, when someone (else in the big team) creates a new executable project with the same kind, he or she has to remember to link the same post-build script otherwise the executable wouldn't be able to find the native DLL. This is what I really don't like, people tend to forget.

2. I can add the native DLL to the project that uses it and in the DLL's property configuration, set it to "Copy If Newer". This way, the native DLL will be copied to the project's output folder and to every project that refers to the project. This way, I don't have to remember anything. The native DLL will be copied to the local folder of every dependent project.

This seems a good solution. But msbuild command seems not being able to handle this situation cleverly. For example, suppose project A directly uses native DLL and I add the native DLL to project A. Project B refers to project A and project C refers to project B and project A. if using msbuild to build the solution, the native DLL will be copied repeatedly to the output folder of project C 3 times, one for reference to project A, one for reference to project B, one for reference of project B to project A. In my big solution, towards the end of the dependency link, the native DLL will be copied exponentially many times to the same output folder, regardless of the "Copy If Newer" setting. This takes up tremendous amount of time to build the whole solution.

Now I totally have no idea what is the best solution for my situation. For anyone who uses native DLLs, how do you deploy the DLL so 1. it is convenient for both deploying on developer machine (compile and run) and on user machine (install and run), 2. developers in big team don't have to remember anything when he or she adds new project/installer to the solution, 3. smart enough build manner that avoids unnecessary redundant actions. Thank you for any hint, tutorial on Web, suggestion or clever teach in advance.

Answer 1

What we've done in general is to make all \\bin directories into symlinks to a common directory. This avoids all the transitive copies that VS does for all references, and can speed up a clean build of a large solution up to a factor of 10-20.

We use the following powershell script to set up the symlinks:

param($linkTarget={throw "Link target must be specified"}, $rootDir=".")

ls $rootDir -Recurse -Include *.csproj | % { $_.DirectoryName } | % { Join-Path $_ -ChildPath "bin" } | % {

  Write-Host "Creating link from $_ to $linkTarget"
  if (Test-Path $_)
  {
    Remove-Item -Force -Recurse $_
    cmd /c rd $_
  }

  cmd /c mklink /D $_ $linkTarget
}

Symbolic links require Vista or Win7; if on XP, junctions can be used instead by replacing the call to mklink with a call to junction.exe.

By default, this must be run as administrator.

Answer 2

For starters, avoid using post build events unless there's no other choice. Post build events unmanaged nature tend to fail builds and have low maintainability.

"Copy If Newer" flag may have lame effectiveness, but it will assure you that: 1) the required dependencies will be copied to output 2) it won't fail the build 3) has virtually no maintainability.

Secondly, you can reduce redundant copying by setting only the last project in the build chain with the "Copy If Newer" flag, eg: project A directly refers the DLL but does not copy it to output, project C refers indirectly to project A through project B, and has dummy reference to the DLL with "Copy If Newer" enabled.

Answer 3

Steve,

I add native dlls to my projects all the time and I haven't had a problem with adding the dll as content to the project. You can actually add the necessary dll as a project link rather than actual content. This way, the dll will not be copied into the project folder as well as into the target folder.

Here's an explanation from devx:

To add a shared file, open the dialog to select an existing file with the Project | Add Existing Item menu item and select the file you want to include. Then, instead of clicking the Open button, click the arrow on the left of that button, and click Link File from the list that drops down. This way you link to the original file, not to a local copy of it.

Do you build all the projects into a common solution target directory, ie ..\\bin\\debug? This may curtail unnecessary copies.

Adding the project link to the dll is definitely easier to maintain than copying in a build event.

Answer 4

There must be something "dodgy" in your project setup (like a circular reference) to cause an infinite copy loop - but finding it could be tricky and/or time consuming.

Two things I would try:

• Turn off the "newer" flag and see if it has any effect on your problem. Copying a single dll is unlikely to impact build times very much, and problems can come about with this option when file datestamps get confused. (I wouldn't expect this to cause repeated copy attempts though)

• Try using a post-build event with xcopy to copy the file instead of relying on the automatic system. Then you'll be in full control of a simple system instead of wondering why a clever system doesn't work.

Answer 5

Personally, I'd treat the native DLL and assembly DLL as one entity. Adding a Link to the file is one way. Alternatively, you could have a separate main build script that massages your project and installer.

For instance, I have a Library.msbuild file that runs a MsBuild task of a source .csproj as my Build target. My Deploy target uses a copy task to copy the dlls from source\\library\\LibraryName\\bin\\Configuration*.dll to library\\LibraryName. I have one library location for all the source files I need to build and the other is all the bins that my main project references.

Here's the entire Library.msbuild script:`

<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
    <PropertyGroup>
        <Configuration Condition=" '$(Configuration)' == '' ">Release</Configuration>
        <ProjectDirectory>library\AspNetMvc\DataAnnotationsModelBinder\src</ProjectDirectory>
        <LibraryDirectory>..\library\AspNetMvc</LibraryDirectory>
    </PropertyGroup>
    <ItemGroup>
       <CompiledBinaries Include="$(ProjectDirectory)\bin\$(Configuration)\*.dll" />
    </ItemGroup>

    <Target Name="Clean">
        <MSBuild Projects="$(ProjectDirectory)\Microsoft.Web.Mvc.DataAnnotations.csproj"
        Targets="Clean" Properties="Configuration=$(Configuration)" />
    </Target>

    <Target Name="PreBuild" DependsOnTargets="Clean">
    </Target>

    <Target Name="Build" DependsOnTargets="PreBuild">
        <MSBuild Projects="$(ProjectDirectory)\Microsoft.Web.Mvc.DataAnnotations.csproj"
        Targets="Build" Properties="Configuration=$(Configuration)" />
    </Target>

    <Target Name="Deploy" DependsOnTargets="Build">
        <Copy SourceFiles="@(CompiledBinaries)" DestinationFolder="$(LibraryDirectory)"/>
    </Target>
</Project>  

` In your instance, I would move the Deploy copy task to PreBuild or remove it entirely to depend on the Add Existing method copying the dll. The goal is that when you run\\debug your project that everything will be bin relative. You could even have bin\\win32 if you're separating native from .NET DLLs.

Every build is now done from a .build.cmd batch file (1-click build) that is simply this: C:\\Windows\\Microsoft.NET\\Framework\\v3.5\\MSBuild Library.msbuild /t:Deploy . You're free to add more targets in the chain like test after build but before deploy.

If you are using WiX or any other type of installer, you should be able to use relative paths for the files themselves. I have a top-level staging\\ directory that I have no problem getting into from source\\install\\ (..\\..\\staging\\ icky, yes but works). Worst case you can place your staging directory somewhere inside the install path.

In most instances you can get away with not having to abstract MsBuild further than your .csproj or .wixproj files. There's a very low ceiling that Before/AfterBuild runs into but I find myself opting for this approach in almost every situation now.

Answer 6

See this post: http://blog.alexyakunin.com/2009/09/making-msbuild-visual-studio-to.html

It explains how to automatize copying of N-th level dependencies into Bin folder.

If you have some assembly, that isn't referenced even indirectly, you can create another one (eg MyProject.ThirdPartyMagnet ), reference this assembly from this project and reference MyProject.ThirdPartyMagnet from the project you need to copy all the assemblies to.