Passing structure to unmanaged code from C#

Question

I've been fighting this problem for some time now and am hoping someone can help. I'm converting a VB6 application to C#. Everything I'm about to show works perfect in VB6, but fails in C#.

I'm making an API call into a C dll, passing a structure. The structure gets values modified in the C side and I should see the modified values in my return structure.

Note, Sig is an int, Status is an int, and CLIENTID is a UINT_PTR.

Here are the C structures:

typedef struct
{
    short               vers;               
    short               flags;              
    short               cmd;                
    short               objtype;            
    DWORD               id;                 
    DWORD               client;             
    char                buf[MAX_TOOLBUF];   
    DWORD               toolID;             
    NMSG                msg;                
    DWORD               caller;             
    CLIENTID                    clientID;           
    DWORD               ticket; 
    ToolResult PTR                      result;     
        long                spare[4];           
} Request;

typedef struct
{
    DWORD       hwnd;
    DWORD       message;
    DWORD       wParam;
    DWORD       lParam;
} NMSG;

typedef struct
{
Sig          sig;               
short               cnt;                
Status              error;              
DWORD               ticket;             
DWORD               toolID;             
long                spare[4];           
} ToolResult;

In my C# code, I have the following structs defined to map to the C structs above:

[StructLayout(LayoutKind.Sequential)]
public struct NMSG
{
    public int hWnd;
    public int msg;
    public int wParam;
    public int lParam;
}

[StructLayout(LayoutKind.Sequential)]
public struct Request
{
    public Int16 vers;        
    public Int16 flags;        
    public Int16 cmd;         
    public int16 objType;     
    public int id;             
    public int Client;         

    [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 260)]
    public string buf;

    public int toolID;         
    public NMSG msg;           
    public int caller;        
    public IntPtr clientID;       
    public int ticket;         
    public ToolResult result;         

    [MarshalAs(UnmanagedType.ByValArray, SizeConst= 4) ]
    public int64[] spare;       
}

This is the method decoration in the C code:

SendRequest(Request PTR req)
{
    ....
}

Here is my PInvoke declaration in C# for the API in C that uses these structures:

    [DllImport("TheCDLL.dll", EntryPoint = "_Request@4", CharSet = CharSet.Ansi, CallingConvention = CallingConvention.Cdecl)]
    public static extern void Request(ref Request req);

And here is my C# code that fills the structure and makes the API call:

Request req = new Request();

req.vers = 1;
req.toolID = 4000;
req.Client = 0;
req.cmd = 11;
req.objType = 1;;
req.id = 1;
req.msg.hWnd = Hwnd; // verified this is a valid window handle
req.msg.msg = 1327;
req.msg.wParam = 101;
req.msg.lParam = 0;
req.spare = new int[4];
req.buf = new string(' ', 260);
req.flags = 11;

Request(ref req);

The problem is, in the VB6 code the structure item 'result' gets filled with a value after I make the API call. In C#, 'result' is always just 0. I'm guessing I must have something wrong with the way I'm marshaling? I've read dozens of articles and have tried lots of different ways to get this working with no success. Any help is appreciated!

UPDATE: Now that I've updated the code based on the suggestions below (fixing types and cleaning up the code as originally written), I'm getting an exception:

System.AccessViolationException: Attempted to read or write protected memory. This is        often an indication that other memory is corrupt.

I've found that this happens as soon as I changed to 'public Int16 vers' from 'public int vers' in my Request structure. Thoughts?

Answer 1

UnmanagedType.Struct actually isn't for substructures; it makes the Marshaler pass the related object as a VARIANT. That isn't what you want here. Try taking off all the MarshalAs statements except for strings (for the length), arrays (also for the length), and enumerated types (I don't see any here, but in my DLLs they've been UnmanagedType.I4) and see if that works.

There's also some extra things in your C# struct that aren't in your C version, a couple variables don't match between the two, and your given C function doesn't return anything but your C# extern is expecting an IntPtr. These do need to match for everything to go smoothly. (I'm guessing it's just simplification for the site, but it is something to keep in mind.)