C# pass pointer to struct (containing non-blittable types) to unmanaged C++ DLL

Question

I try to use a C++ DLL in my C# code.
All data that is read must be passed to the dll as a pointer to a struct. My first idea was to just reserve some unmanaged memory, pass the pointer to the function and extract the data afterwards. The problem is that the functions will only returns an error code that translates to "An argument is not valid".

From the DLLs Header (Result and BusPortId shortened)

typedef enum {
    BUS_COM1 = 0x01,  // 2 to 15 ommited
    BUS_COM16 = 0x10,
    BUS_USB1 = 0x101,  // 2 to 15 ommited
    BUS_USB16 = 0x110
} BusPortId;

typedef enum {
    BUS_SUCCESS    = 0,     //!< The operation was completed successfully
    BUS_ERROR      = 0x100,  //!< An error occured
    BUS_INVALIDARG = 0x1000, //!< An argument is not valid
} Result


struct BusPortInfo
{
    ULONG       portInfoSize;
    CHAR        portText[64];
    BOOL        portLocked;
    BusPortId   portId;
};

Result BUSDRV_API busGetAvailablePortCount( ULONG *retCount );

Result BUSDRV_API busGetAvailablePort( ULONG index, BusPortInfo *portInfo );

The relevant parts of my C# program so far

enum BusPortId
{
    BUS_COM1 = 0x01,  // 2 to 15 ommited
    BUS_COM16 = 0x10,
    BUS_USB1 = 0x101,  // 2 to 15 ommited
    BUS_USB16 = 0x110
};

public enum Result
{
    BUS_SUCCESS = 0,       //!< The operation was completed successfully
    BUS_ERROR = 0x100,  //!< An error occured
    BUS_INVALIDARG = 0x1000, //!< An argument is not valid
};

struct BusPortInfo
{
    public ULONG portInfoSize;
    unsafe public fixed char portText[64];
    public BOOL portLocked;
    public BusPortId portId;
}

[DllImport(DLL_Path)]
unsafe static extern Result busGetAvailablePortCount(ULONG* retCount);
[DllImport(DLL_Path)]
unsafe static extern Result busGetAvailablePort(ULONG index, BusPortInfo* portInfo);

ulong count= 0;
Result res = busGetAvailablePortCount(&count);

ulong index = 0;
BusPortInfo info = new BusPortInfo();
Result res = busGetAvailablePort(0, &info);

The call to busGetAvailablePortCount (and other similar functions) works without any problems. But when I call busGetAvailablePort I get the following in my console output

Cannot marshal 'parameter #2': Pointers cannot reference marshaled structures. Use ByRef instead.

The Problem is that I can change my struct in C# so that I can pass the pointer, but then the function from the DLL also returns "An argument is not valid"

What do I have to do to my struct so I can pass a pointer to it to a function while still getting accepted by the DLL?

PS Sorry for the bad english, I'm not a native speaker.

Answer 1

There are a lot of problems with these declarations, tends to happen when the programmer keeps hacking the code to try to make the pinvoke call work. The most likely correct declaration for the structure is:

    [StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi)]
    struct BusPortInfo {
        public int portInfoSize;
        [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 64)]
        public string portText;
        public bool portLocked;
        public BusPortId portId;
    }

Emphasizing that an ULONG in native code is a 32-bit type and fixed just isn't necessary and is pretty awkward. This struct is not blittable due to the bool and string member, nothing much to fret about.

The [DllImport] declaration needs to declare the 2nd argument correctly. The CallingConvention property always matters a great deal and we can't see what BUSDRV_API means. Punting:

    [DllImport(DLL_Path, CallingConvention = CallingConvention.StdCall)]
    static extern Result busGetAvailablePortCount(out int retCount );

    [DllImport(DLL_Path, CallingConvention = CallingConvention.StdCall)]
    static extern Result busGetAvailablePort(int index, 
                            [In, Out] ref BusPortInfo portInfo);

And the call does not look correct. When a struct has a "size" member then the api contract normally demands that it is set before the call. It is a safety measure, it ensures that the api cannot corrupt memory when the caller uses the wrong structure declaration. And when set wrong, an "Invalid argument" error is the expected outcome. So write it similar to:

int count;
Result res = busGetAvailablePortCount(out count);
if (res != Result.BUS_SUCCESS) throw new Exception("Api failure " + res.ToString());

for (int ix = 0; ix < count; ++ix) {
    BusPortInfo info;
    info.portInfoSize = Marshal.SizeOf(typeof(BusPortInfo));   // Important!
    res = busGetAvailablePort(ix, ref info);
    if (res != Result.BUS_SUCCESS) throw new Exception("Api failure " + res.ToString());
    // etc...
}

Not tested of course, should be in the ballpark. If you still have problems then verify that sizeof(BusPortInfo) in native code matches the value of Marshal.SizeOf(typeof(BusPortInfo)) in C#. And if all fails then use C++/CLI instead so you can use the native declarations directly. And talk to the owner of the DLL for proper usage instructions, preferably he'll write a pinvoke sample for you.