How can I use EnvDTE as a part of a VS2015 project/solution in order to change its own settings without using VS extensions?

Question

Update: I've developed my own PowerShell solution (in the answers), but I'd still like to know if anyone knows of a NuGet package that mimics the behavior of the abandoned StudioShell project.

I'm trying to use EnvDTE to make some problematic settings persistent (no matter what future devs do) in my Visual Studio 2015 solutions/projects using only pre/post-build scripts, NuGet packages, or some built-in feature of VS. ( I'll refer to these settings/configuration changes as "tweaks" from here on )

The end goals are:

1. No external tools/extensions required - Whatever I use to make these changes should be a standard part of Visual Studio 2015 and/or should be portable with the solution, or it should be reasonable to expect it to be part of recent Windows operating systems (Windows 7+).

   In other words, future developers who inherit my work should be able to pull the solution from source control and build it without needing to install VS extension "X", make change "Y" in their build environment, or change "Z" in their user specific project settings.

2. Discoverable - The component/tool/method used to make the tweaks should be exposed somewhere in the Visual Studio interface without digging further than the solution tree (including context menus, project properties, etc.) - so the automated tweaks can be altered or disabled without too much digging.

   I don't mind the GUI element just being a link to some code. I expect the future devs can figure out the code, but they need to be able to easily discover that this code is responsible for maintaining the EnvDTE tweaks.

   For example, custom MSBuild tasks could potentially be considered a "built-in" solution, but those custom tasks are completely hidden to the average developer in my office. ( Pre/Post build events are borderline hidden too, but at least they are in the GUI where devs can run across those by accident on occasion )

So, does anyone know of a NuGet package ( StudioShell may have been perfect, but it looks like that project has been abandoned and there's no support for VS2015 ) or some other solution that would give me access to a DTE object bound to the current instance of VS from within a pre/post-build process? Or better yet, a method of using EnvDTE to tweak things when the solution/project is loaded?

---

Specific tweaks

The specific settings I'm trying to tweak from EnvDTE are not really relevant in my opinion, but since I know someone will ask, here are the tweaks I'm currently trying to get in place during solution/project load or at least before the build/debug process starts.

1. Set the startup project - I'm surprised this bug still plagues me, but despite having already set the startup project in the solution, inevitably some other dev pulls down my solution and ends up with the wrong project set as the startup project which messes with the dependencies/build order and breaks the build.

2. Configure user/environment specific project settings (ie. settings stored in <project>.user files) that are known to be generic enough for all build environments to use.

   More specifically, I want to set the Project Properties -> Web -> Start Action configuration items in a web project. For normal web projects I understand why these settings are user specific because who knows what browsers the dev might have. But, I have a web hosted WCF service with test scripts nested under the project directory - so project debugging should always be able to be done running those scripts using relative paths known at build time even when the path to the solution/project changes between environments.

Answer 1

Here's the post-build event PowerShell solution I've come up with (a workaround for users running in a Restricted execution policy is at the end of this answer).

1) Create a PowerShell script containing the following:

Param(
    [Parameter(Mandatory=$True,Position=1)]
    [string] $projPath,
    [Parameter(Mandatory=$True,Position=2)]
    [string] $debugScriptPath
)

# Setup new data types and .NET classes to get EnvDTE from a specific process
Add-Type -TypeDefinition @"
using System;
using System.Diagnostics;
using System.Runtime.InteropServices;
using System.Runtime.InteropServices.ComTypes;
using System.Text.RegularExpressions;

public static class NTDLL
{
    [StructLayout(LayoutKind.Sequential)]
    public struct PROCESS_BASIC_INFORMATION
    {
        public IntPtr Reserved1;
        public IntPtr PebBaseAddress;
        public IntPtr Reserved2_0;
        public IntPtr Reserved2_1;
        public IntPtr UniqueProcessId;
        public UIntPtr ParentUniqueProcessId;
    }

    public static UInt32 GetParentProcessID(IntPtr handle)
    {
        PROCESS_BASIC_INFORMATION pbi = new PROCESS_BASIC_INFORMATION();
        Int32 returnLength;
        UInt32 status = NtQueryInformationProcess(handle, IntPtr.Zero, ref pbi, Marshal.SizeOf(pbi), out returnLength);

        if (status != 0)
            return 0;

        return pbi.ParentUniqueProcessId.ToUInt32();
    }

    [DllImport("ntdll.dll")]
    private static extern UInt32 NtQueryInformationProcess(IntPtr processHandle, IntPtr processInformationClass, ref PROCESS_BASIC_INFORMATION processInformation, Int32 processInformationLength, out Int32 returnLength);
}

public static class OLE32
{
    [DllImport("ole32.dll")]
    public static extern Int32 CreateBindCtx(UInt32 reserved, out IBindCtx ppbc);
}

public class VisualStudioProcBinder
{
    public static Object GetDTE(int processId)
    {
        Regex VSMonikerNameRegex = new Regex(@"!?VisualStudio\.DTE([\.\d]+)?:" + processId);

        object runningObject = null;

        IBindCtx bindCtx = null;
        IRunningObjectTable rot = null;
        IEnumMoniker enumMonikers = null;

        try
        {
            Marshal.ThrowExceptionForHR(OLE32.CreateBindCtx(0, out bindCtx));
            bindCtx.GetRunningObjectTable(out rot);
            rot.EnumRunning(out enumMonikers);

            IMoniker[] moniker = new IMoniker[1];
            IntPtr numberFetched = IntPtr.Zero;
            while (enumMonikers.Next(1, moniker, numberFetched) == 0)
            {
                IMoniker runningObjectMoniker = moniker[0];

                string name = null;

                try
                {
                    if (runningObjectMoniker != null)
                        runningObjectMoniker.GetDisplayName(bindCtx, null, out name);
                }
                catch (UnauthorizedAccessException)
                {
                    // Do nothing, there is something in the ROT that we do not have access to.
                }

                if (!string.IsNullOrEmpty(name) && VSMonikerNameRegex.IsMatch(name))
                {
                    Marshal.ThrowExceptionForHR(rot.GetObject(runningObjectMoniker, out runningObject));
                    break;
                }
            }
        }
        finally
        {
            if (enumMonikers != null)
                Marshal.ReleaseComObject(enumMonikers);

            if (rot != null)
                Marshal.ReleaseComObject(rot);

            if (bindCtx != null)
                Marshal.ReleaseComObject(bindCtx);
        }

        return runningObject;
    }
}
"@


# Get the devenv.exe process that started this pre/post-build event
[Diagnostics.Process] $dteProc = [Diagnostics.Process]::GetCurrentProcess();

while ($dteProc -and $dteProc.MainModule.ModuleName -ne 'devenv.exe')
{
    #Write-Host "$(${dteProc}.Id) = $(${dteProc}.MainModule.ModuleName)";
    try { $dteProc = [Diagnostics.Process]::GetProcessById([NTDLL]::GetParentProcessID($dteProc.Handle)); }
    catch { $_; $dteProc = $null; }
}

# Get dteCOMObject using the parent process we just located
$dteCOMObject = [VisualStudioProcBinder]::GetDTE($dteProc.Id);

# Get the project directory
$projDir = Split-Path $projPath -Parent;

# If the script path does not exist on its own - try using it relative to the project directory
if (!(Test-Path $debugScriptPath)) {
    $debugScriptPath = "${projDir}\${debugScriptPath}";
}

#####################################################
# Finally, tweak the project
#####################################################
if ($dteCOMObject) {

    # Get the project reference from DTE
    $dteProject = $dteCOMObject.Solution.Projects | ? { $_.FileName -eq $projPath } | Select-Object -First 1;

    # Set this project as the startup project
    $startupProj = $dteCOMObject.Solution.Properties["StartupProject"].Value;

    if ($startupProj -ne $dteProject.Name) {
        $dteCOMObject.Solution.Properties["StartupProject"].Value = $dteProject.Name;
    }

    # Get the external debug program and arguments currently in use
    $debugProg = $dteProject.Properties['WebApplication.StartExternalProgram'].Value;
    $debugArgs = $dteProject.Properties['WebApplication.StartCmdLineArguments'].Value;

    # If an external debug program is not set, or it is set to cmd.exe /C "<file path>"
    # and "file path" points to a file that doesn't exist (ie. project path has changed)
    # then correct the program/args
    if (!$debugProg -or ($debugProg -eq $env:ComSpec -and $debugArgs -match '^\s*/C\s+("?)([^"]+)\1$'-and !(Test-Path $Matches[2]))) {
        if (!$debugProg) { $dteProject.Properties['WebApplication.DebugStartAction'].Value = 2;  } # 2 = run external program

        $dteProject.Properties['WebApplication.StartExternalProgram'].Value = $env:ComSpec; # run cmd.exe

        # pass "<project dir>\Testing\Debug.cmd" as the program to run from cmd.exe
        $dteProject.Properties['WebApplication.StartCmdLineArguments'].Value = "/C `"${debugScriptPath}`"";
    }

    # Release our COM object reference
    [Runtime.InteropServices.Marshal]::ReleaseComObject($dteCOMObject) | Out-Null;
}

2) Call the PowerShell script from your project post-build like:

powershell.exe -File "$(ProjectDir)script.ps1" "$(ProjectPath)" "$(ProjectDir)Testing\Debug.cmd"

The first parameter (after -File ) is the path to the script you created in step 1, the second parameter is the path to the project being built, and the third parameter (which your script will probably not have unless you're trying to do exactly what I am) is the path to the batch file/script that to be configured to run when debugging with an external program.

---

Workaround for users limited to running under a Restricted execution policy

(ie. powershell.exe -Command "Set-ExecutionPolicy Unrestricted" does not work)

If PowerShell is locked into the Restricted execution policy you will not be able to run a PowerShell script either using powershell.exe -File script.ps1 commands or through dot-sourcing methods like powershell.exe -Command ". .\\script.ps1" . However, I've discovered that you can read a script into a variable and then run Invoke-Expression $ScriptContent . (Seems odd to me that this works, but it does)

The workaround consists of:

1) Create a PowerShell script using the same content from above, but exclude the Param(...) lines at the top.

2) Call the PowerShell script from your project post-build like:

powershell -Command "& { $projPath='$(ProjectPath)'; $debugScriptPath='$(ProjectDir)Testing\Debug.cmd'; Get-Content '$(ProjectDir)script.ps1' -Encoding String | ? { $_ -match '^^\s*[^^#].*$' } | %% { $VSTweaks += $_ + """`r`n"""; }; Invoke-Expression $VSTweaks; } }"

This reads contents of script.ps1 into a variable named $VSTweaks (skipping lines that are only comments - ie. digital signature lines which cause problems in some scenarios), and then runs the script contents with Invoke-Expression . The values that were passed into $projPath and $debugScriptPath through parameters in the original script are now set at the beginning of the call in powershell.exe -Command "& { $projPath ...} . (If they are not set then the script will fail)

NOTE: Because the VS project post-build event contents are executed as a Windows batch file you have to escape a lot of characters that are special to Windows batch files. That explains some of the confusing character combinations in the script

• % = %%
• ^ = ^^
• " = """ (I'm honestly not sure why this is required, but it seems to be)

---

Tip

I've started wrapping the entire PowerShell call ( <PowerShell commands> in the example below) in try catch statements in order to make any PowerShell errors show up in the Visual Studio "Error List..." view when a build fails.

powershell -Command "& { try { <PowerShell commands> } catch { Write-Host "post-build : PowerShell error $^($_.Exception.HResult^) : $_"; exit $_.Exception.HResult; } }"

The resulting error message in VS looks like this:

Answer 2

You can run a VBScript file (.vbs) with code like this:

Dim dte
Set dte = GetObject(, "VisualStudio.DTE")
dte.ExecuteCommand("Help.About")

Answer 3

While I was trying to find a solution I briefly looked into making my own .NET PowerShell cmdlet that would return the EnvDTE.DTE object for the instance of VS that started the PowerShell script.

The code is ugly, but it works and it met my requirements for the most part (more on that in the notes at the end). So, I thought I'd go ahead and include it as an answer as well just in case someone prefers this method or needs a starting point for making their own DTE cmdlet.

Usage Example:

PS C:\> Import-Module .\GetDTECmdlet.dll;
PS C:\> $dte = Get-DTE | Select-Object -First 1;
PS C:\> $dte = Get-DTE -ProcID 8547 | Select-Object -First 1;
PS C:\> $dte = Get-DTE -FromAncestorProcs | Select-Object -First 1;
PS C:\> $dte.ExecuteCommand('Help.About');
PS C:\> [Runtime.InteropServices.Marshal]::ReleaseComObject($dte); | Out-Null;

---

Cmdlet Parameters:

-ProcID <int> : int = the process ID (PID) of a running instance of Visual studio to get the DTE from

-FromAncestorProcs : If this switch is specified then the Get-DTE will limit its search of DTE objects to Visual Studio processes higher up in the process tree (ie. parent/ancestor processes) from the PowerShell session that called it.

---

Source Code:

(Make sure your project references System.Management.Automation.dll and envdte.dll )

using System;
using System.Diagnostics;
using System.Collections.Generic;
using System.Text.RegularExpressions;
using System.Management.Automation;
using System.Runtime.InteropServices;
using System.Runtime.InteropServices.ComTypes;

namespace VSAutomation
{
    [Cmdlet(VerbsCommon.Get, "DTE")]
    [OutputType(typeof(EnvDTE.DTE))]
    public class GetDTECmdlet : Cmdlet, IDisposable
    {
        private Int32 procID = -1;
        private IBindCtx bindCtx = null;
        private IRunningObjectTable rot = null;
        private IEnumMoniker monikerEnumerator = null;
        private IMoniker[] moniker = new IMoniker[1];
        private ProcCollection matchingProcs = new ProcCollection();

        [Parameter]
        public SwitchParameter FromAncestorProcs { get; set; }

        [Parameter]
        public Int32 ProcID { get { return procID; } set { procID = value; } }

        protected override void BeginProcessing()
        {
            base.BeginProcessing();

            Marshal.ThrowExceptionForHR(OLE32.CreateBindCtx(0, out bindCtx));
            bindCtx.GetRunningObjectTable(out rot);
            rot.EnumRunning(out monikerEnumerator);
        }

        protected override void ProcessRecord()
        {
            base.ProcessRecord();

            Regex VSMonikerNameRegex = new Regex(@"^!?VisualStudio\.DTE([\.\d]+)?:(?<PID>\d+)$", RegexOptions.Compiled | RegexOptions.IgnoreCase);

            // Get a list of ancestor PIDs if the results should be limited based on ancestor processes
            if (FromAncestorProcs.IsPresent)
            {
                try
                {
                    using (Process thisProc = Process.GetCurrentProcess())
                    {
                        Process proc = thisProc;
                        Int32 parentProcID;

                        while ((parentProcID = NTDLL.GetParentProcessID(proc.Handle)) != 0)
                        {
                            proc = Process.GetProcessById(parentProcID);
                            matchingProcs.Add(new ROTProc(proc));
                        }
                    }
                }
                catch { }
            }

            // Loop through the running objects and find a suitable DTE
            while (monikerEnumerator.Next(1, moniker, IntPtr.Zero) == 0)
            {
                Object runningObject;
                IMoniker runningObjectMoniker = moniker[0];

                if (!FromAncestorProcs.IsPresent && ProcID == -1)
                {
                    // Returning all DTE objects from running processes

                    //Only return each object once
                    if (!matchingProcs.Contains(runningObjectMoniker))
                    {
                        Marshal.ThrowExceptionForHR(rot.GetObject(runningObjectMoniker, out runningObject));
                        EnvDTE.DTE dte = runningObject as EnvDTE.DTE;

                        if (dte != null && !matchingProcs.Contains(dte))
                        {
                            matchingProcs.Add(new ROTProc(dte, runningObjectMoniker));
                            WriteObject(runningObject);
                        }
                    }

                    continue;
                }


                // Returning only DTE objects from ancestor processes or a specific process
                Match nameMatch;
                String name = null;

                try
                {
                    if (runningObjectMoniker != null)
                        runningObjectMoniker.GetDisplayName(bindCtx, null, out name);
                }
                catch (UnauthorizedAccessException)
                {
                    // Do nothing, there is something in the ROT that we do not have access to.
                }

                if (String.IsNullOrEmpty(name))
                    continue;

                nameMatch = VSMonikerNameRegex.Match(name);

                if (!nameMatch.Success)
                    continue;

                if (ProcID != -1)
                {
                    if (Int32.Parse(nameMatch.Groups["PID"].Value) != ProcID)
                        continue;

                    //Found a match for the specified process ID - send it to the pipeline and quit enumerating
                    Marshal.ThrowExceptionForHR(rot.GetObject(runningObjectMoniker, out runningObject));
                    if (runningObject is EnvDTE.DTE)
                    {
                        WriteObject(runningObject);
                        return;
                    }
                }

                // collect DTE objects so that they can be returned in order from closest ancestor to farthest ancestor in the event that VS launched VS which launched MSBUild ...
                ROTProc ancestorProc = matchingProcs.GetByProcId(Int32.Parse(nameMatch.Groups["PID"].Value));

                if (ancestorProc == null)
                    continue;

                Marshal.ThrowExceptionForHR(rot.GetObject(runningObjectMoniker, out runningObject));
                ancestorProc.DTE = runningObject as EnvDTE.DTE;
            }

            if (!FromAncestorProcs.IsPresent)
                return;

            for (Int32 i = 0; i < matchingProcs.Count; i++)
                if (matchingProcs[i].DTE != null) WriteObject(matchingProcs[i].DTE);
        }

        protected override void EndProcessing()
        {
            base.EndProcessing();
            Dispose();
        }

        protected override void StopProcessing()
        {
            base.StopProcessing();
            Dispose();
        }

        public void Dispose()
        {
            if (monikerEnumerator != null)
            {
                Marshal.ReleaseComObject(monikerEnumerator);
                monikerEnumerator = null;
            }

            if (rot != null)
            {
                Marshal.ReleaseComObject(rot);
                rot = null;
            }

            if (bindCtx != null)
            {
                Marshal.ReleaseComObject(bindCtx);
                bindCtx = null;
            }

            if (matchingProcs != null)
            {
                matchingProcs.Dispose();            
                matchingProcs = null;
            }
        }

        private class ROTProc : IDisposable
        {
            public Process Proc = null;
            public EnvDTE.DTE DTE = null;
            public IMoniker Moniker = null;
            public IntPtr COMPtr = IntPtr.Zero;

            public ROTProc(Process Proc, EnvDTE.DTE DTE = null, IMoniker Moniker = null)
            {
                this.Proc = Proc;
                this.DTE = DTE;
                this.Moniker = Moniker;

                if (DTE != null)
                    COMPtr = Marshal.GetComInterfaceForObject(DTE, typeof(EnvDTE._DTE));
            }

            public ROTProc(EnvDTE.DTE DTE, IMoniker Moniker) : this(null, DTE, Moniker) { }

            public void Dispose()
            {
                if (Proc != null)
                {
                    try { Proc.Dispose(); }
                    catch (ObjectDisposedException) { }

                    Proc = null;
                }

                if (COMPtr != IntPtr.Zero)
                {
                    try { Marshal.Release(COMPtr); }
                    catch { }

                    COMPtr = IntPtr.Zero;
                }
            }
        }

        private class ProcCollection : System.Collections.CollectionBase, IDisposable
        {
            public ROTProc this[Int32 index]
            {
                get { return InnerList[index] as ROTProc; }
                set { InnerList[index] = value; }
            }

            public Int32 Add(ROTProc p)
            {
                return InnerList.Add(p);
            }

            public Boolean Contains(IMoniker Moniker)
            {
                if (Moniker == null)
                    return false;

                foreach (ROTProc p in this)
                    if (p != null && Moniker.IsEqual(p.Moniker) == 0) return true;

                return false;
            }

            public Boolean Contains(EnvDTE.DTE DTE)
            {
                if (DTE == null)
                    return false;


                foreach (ROTProc p in this)
                {
                    if (p != null && (
                        Marshal.Equals(DTE, p.DTE) ||
                        Marshal.GetComInterfaceForObject(DTE, typeof(EnvDTE._DTE)) == p.COMPtr))
                    {
                        return true;
                    }
                }


                return false;
            }

            public ROTProc GetByProcId(Int32 ProcId)
            {
                foreach (ROTProc p in this)
                    if (p != null && p.Proc != null && p.Proc.Id == ProcId) return p;

                return null;
            }

            public void Dispose()
            {
                foreach (ROTProc p in this)
                {
                    try { if (p != null) p.Dispose(); }
                    catch (ObjectDisposedException) { }
                }
            }
        }
    }

    #region Supporting interop classes

    public static class OLE32
    {
        [DllImport("ole32.dll")]
        public static extern Int32 CreateBindCtx(UInt32 reserved, out IBindCtx ppbc);
    }

    public static class NTDLL
    {
        [StructLayout(LayoutKind.Sequential)]
        public struct PROCESS_BASIC_INFORMATION
        {
            public IntPtr Reserved1;
            public IntPtr PebBaseAddress;
            public IntPtr Reserved2_0;
            public IntPtr Reserved2_1;
            public IntPtr UniqueProcessId;
            public IntPtr ParentUniqueProcessId;
        }

        public static Int32 GetParentProcessID(IntPtr handle)
        {
            PROCESS_BASIC_INFORMATION pbi = new PROCESS_BASIC_INFORMATION();
            Int32 returnLength;
            UInt32 status = NtQueryInformationProcess(handle, IntPtr.Zero, ref pbi, Marshal.SizeOf(pbi), out returnLength);

            if (status != 0)
                return 0;

            return pbi.ParentUniqueProcessId.ToInt32();
        }

        [DllImport("ntdll.dll")]
        private static extern UInt32 NtQueryInformationProcess(IntPtr processHandle, IntPtr processInformationClass, ref PROCESS_BASIC_INFORMATION processInformation, Int32 processInformationLength, out Int32 returnLength);
    }

    #endregion Supporting interop classes
}

NOTE: I still prefer the single PowerShell script solution to this one (posted in another answer ). But that's mainly because of the Restricted execution policy that is enforced in my environment. Because of that, even when I was using this cmdlet to do the heavy lifting of getting the DTE reference, the rest of my PowerShell code had to be compressed into an ugly, batch escaped, hard to understand, single line of code (ie powershell.exe -Command " ... %% ^^ ... " ).

Even as I was writing it I found it hard to follow. So, I knew any future developers picking up the project would have been cursing me if they ever had to tweak it.

While trying to find a way to get around the "ugly one-liner" problem, I discovered the method of using Get-Content and Invoke-Expression to "psuedo-source" PowerShell scripts. Once I had that routine down I decided I'd rather have a single PowerShell script rather than a project to make a cmdlet and a PowerShell script using it.