Calling a unmanaged C method with a const char** argument from managed C# code

Question

I know that when calling an unmanaged method accepting a char* argument from C#, it is possible to pass a StringBuilder and have the unmanaged C code modify it. You however have to know what size the data is you want put into the StringBuilder, so you can pass a buffer of the correct size. I have found many threads helping with this.

However, I have a C method that accepts a char** argument.

This allows methods like check_if_encrypted (shown below) to provide malloc 'd error messages without the calling method having to know how much space to allocate for the error message buffer, by using something like the following code in check_if_encrypted : *strTheCharStarStarArgument = strLocalMallocdErrorMessage , and calling with by passing &strErrorMessage where strErrorMessage is a char*

What DLLImport signature should be used for such a method in C#?

For example, I have the following C code:

Header:

extern __declspec(dllexport) KeyFile *create_source_key_file_from_path(char *strPath);
extern __declspec(dllexport) int check_if_encrypted(KeyFile *kfSourceKey, int ktSourceKeyType, const char **strErrorMessage);

Main Code:

KeyFile *create_source_key_file_from_path(char *strPath) {
    /* Snip */
}

int check_if_encrypted(KeyFile *kfSourceKey, int ktSourceKeyType, const char **strErrorMessage) {
    /* Snip */
    char* strLocalErrorMessage = (char*)malloc(sizeof(char)*17);
    strcpy(strLocalErrorMessage,"This is an error");
    *strErrorMessage = strLocalErrorMessage;
    /* Snip */
}

/* Snip */

int main(int argc, char *argv[]) {
    KeyFile *kfSource;
    char *strErrorMessage;
    const char **strErrorMessageP = &strErrorMessage;
    int intIsEncrypted;

    kfSource = create_source_key_file_from_path("test.dat");
    intIsEncrypted = check_if_encrypted(kfSource,0,strErrorMessageP);
    if (strErrorMessage != NULL) {
       /* Handle error here. */
    }
    else if (intIsEncrypted == 1) {
       /* Handle encrypted file here. */
    }
    else {
       /* Handle unencrypted file here. */
    }
}

How my attempt at replicating this main method in C# using the check_if_encrypted unmanaged method, has resulted in a AccessViolationException (Attempted to read or write protected memory...) when calling check_if_encrypted.

[DllImport("PPKConverter.exe", SetLastError = true, CallingConvention = CallingConvention.Cdecl, CharSet=CharSet.Ansi)]
public extern static IntPtr create_source_key_file_from_path([MarshalAs(UnmanagedType.LPStr)] StringBuilder strPath);
[DllImport("PPKConverter.exe", SetLastError = true, CallingConvention = CallingConvention.Cdecl, CharSet = CharSet.Ansi)]
public extern static int check_if_encrypted(IntPtr kfSourceKey, int ktSourceKeyType, [MarshalAs(UnmanagedType.LPStr)] ref StringBuilder sbErrorMessage);

[STAThread]
public static void Main(String[] args)
{
    StringBuilder sbPath = new StringBuilder(@"test.dat");
    StringBuilder sbErrorMessage = new StringBuilder();
    IntPtr ipKeyFile = create_source_key_file_from_path(sbPath); // This works
    int intIsEncrypted = check_if_encrypted(ipKeyFile, 0, ref sbErrorMessage); // This throws the exception.
    /* Snip */
}

I think this is due to my use of ref StringBuilder in the DLLImport signature being incorrect. What should be used instead?

Thanks

Answer 1

The string that the unmanaged code returns to the caller is allocated with a call to malloc . As such the p/invoke framework is not capable of deallocating it. You will also need to export a deallocator to avoid leaks. In order to call the function using p/invoke you'll need to do marshalling manually.

[DllImport("PPKConverter.exe", CallingConvention = CallingConvention.Cdecl)]
public extern static int check_if_encrypted(
    IntPtr kfSourceKey, 
    int ktSourceKeyType, 
    out IntPtr sbErrorMessage
);

Convert the IntPtr returned in sbErrorMessage to a string with a call to Marshal.PtrToStringAnsi . And as mentioned above you need to then pass sbErrorMessage back to a deallocator to avoid leaking.

I removed the SetLastError = true because I doubt that you unmanaged function really does call SetLastError .

Since create_source_key_file_from_path receives a string as input then you should declare it like this:

[DllImport("PPKConverter.exe", CallingConvention = CallingConvention.Cdecl, 
    CharSet=CharSet.Ansi)]
public extern static IntPtr create_source_key_file_from_path(string strPath);

Are you sure that you should be linking to functions in a .exe file?

I also wonder whether it is wise to have the unmanaged code allocate the strings with malloc . That forces you to export a deallocator. You might be better asking the caller to allocate the memory. Or if you have to allocate in the unmanaged code, use a shared heap like the COM heap. That way the C# marshaller can deallocate the memory.