How to read command line parameters passed from VB6 IDE in a .NET library?

Question

I have app1.exe (built with VB6) which takes a bunch of command line parameters. It calls lib1.dll - a C#/.NET interop library which needs access to the command line parameters of the EXE. I use Environment.CommandLine to read the command line params and that works great.

When I am debugging app1.exe in the VB6 IDE, I typically go into project properties, Make tab and set the command line parameters textbox manually. However, the .NET library, when called, doesn't see the command line parameters at all. It just sees that the entire thing is running under the VB6.exe process.

That makes sense (since when debugging, it really does run under VB6.exe), but it makes it difficult to debug, since .NET library never gets access to the actual command line parameters.

My question: short of passing up the command line parameters manually to the .NET lib, is there any other way for it to see them when debugging in the VB6 IDE?

Answer 1

This is just core to the way the VB6 IDE is designed. When you debug VB6 code then you don't run a process, you run an interpreter. Which executes p-code, not machine code. It is an accidentally nice feature, it allows debugging both VB6 code and .NET code that you made [ComVisible]. That would not normally be possible, there can only be one debugger attached to a process. This also explains why your .NET debugger sees the VB6.exe process, that's the process that has both the interpreter and the debugger.

So when you are using Environment.CommandLine, you are actually reading the command line that was passed to the VB6 IDE. Normally that's an empty one.

The proper approach is a clear one: don't try to obtain information through a back-door. Just have your VB6 code set properties on your object.

Answer 2

You could probably start VB6 IDE with command line arguments you'd like for a debug run. Although it looks inconvenient (one has to restart IDE before each run). It also shouldn't work if .NET library cares about program name which is part of command line.

Alternative could be to go after modifying command line of running VB6.exe process then. Since the command line is copied into the process's address space on its creation, the process can modify it. In theory. Bad news is that there's no supported way to do this, to my knowledge. And implementing unsupported way may be tricky to do.

Yet another alternative could be for the .NET library to test if it runs under VB6.exe (this should be clear from command line), and obtain data from some alternative storage in such debug scenario.

Edit

The code below alters memory location returned by GetCommandLine WinAPI func. Hopefully, this same func is used by Environment.CommandLine under the hood.

I have not tested the code with .NET library (yet), but you may wish to try it out anyway. In order to give this method a try:

Run VB6 IDE (VB6.exe) with arbitrary, but sufficiently long, string of command line arguments. Arrange to execute SetCmdLineWhenRunInVB6IDE procedure below when running in VB6 IDE. It is intended to replace IDE's command line with values one set up in project parameters. GetCmdLine and SetCmdLine are mere helpers to facilitate exploration.

Option Explicit

Private Declare Function GetCommandLine _
    Lib "kernel32" Alias "GetCommandLineW" () As Long
Private Declare Function lstrlen _
    Lib "kernel32" Alias "lstrlenW" (ByVal lpString As Long) As Long
Private Declare Sub CopyMemory _
    Lib "kernel32" Alias "RtlMoveMemory" (ByVal pDst As Any, ByVal pSrc As Any, ByVal ByteLen As Long)

Public Function GetCmdLine() As String
    Dim hStr As Long
    Dim hLen As Long

    hStr = GetCommandLine
    hLen = lstrlen(hStr)
    If hLen > 0 Then
        GetCmdLine = Space$(hLen)
        CopyMemory StrPtr(GetCmdLine), hStr, (hLen) * 2
    End If
End Function

Public Sub SetCmdLine(sNewCmdLine As String)
    Dim hStr As Long
    Dim hLen As Long
    Dim sBuf As String

    hStr = GetCommandLine
    hLen = lstrlen(hStr)
    If hLen > 0 Then
        Dim i As Long, hEndMark As Long

        sBuf = Space$(hLen)
        hEndMark = IIf(Len(sNewCmdLine) < Len(sBuf), Len(sNewCmdLine), Len(sBuf))

        For i = 1 To hEndMark
            Mid$(sBuf, i) = Mid$(sNewCmdLine, i)
        Next i

        CopyMemory ByVal hStr, StrPtr(sBuf), LenB(sBuf)
    End If
End Sub

Public Sub SetCmdLineFromVBACommand()
    Dim sVBACmdLine As String

    sVBACmdLine = """" & App.Path & "\" & App.EXEName & ".exe"" " & VBA.Command$
    SetCmdLine sVBACmdLine
End Sub