Pointers using a 32 bit DLL in a 64 bit service

Question

Until recently, we were running a service as x86, but changed it to Any CPU to use introduce some new functionality using a 64 bit DLL. One of the third party DLL's left over was 32 Bit. I setup a DLL surrogate for it in the registry, but that is only half the solution.

I suspect that the pointer being made in memory is not accessible because it's no longer run inProc. What I need to know is how do I use the System.Runtime.InteropServices.Marshall object to get access to the returned pointer.

Thanks in advance.

public Image GetThumbnail(string strFilename)
    {
        SeThumbnailExtractor objExtractor = new SeThumbnailExtractor();
        int hImageSE;
        Image objImage = null;
        try
        {
            objExtractor.GetThumbnail(strFilename, out hImageSE);
            IntPtr iPImage = new IntPtr(hImageSE);
            //fails below
            objImage = Image.FromHbitmap(iPImage);
            _ReturnedImage = objImage;
            _SourceFile = strFilename;
            Marshal.FreeHGlobal(iPImage);
        }
        catch (Exception ex)
        {
            _ErrorMsg = "ERROR: " + ex.Message.ToString();
        }

        if (objExtractor != null)
        {
            System.Runtime.InteropServices.Marshal.ReleaseComObject(objExtractor);
            objExtractor = null;
        }

        return objImage;
    }

Answer 1

Running in a surrogate is completely transparent to a properly implemented COM component. The proxy/stub for the component ensures that the arguments and return values of the interface methods are properly serialized by the proxy into an interop data packet so it can be transmitted across the process divide and deserialized again in the stub.

"Properly implemented" is however the rub. If you get a raw bitmap handle or pointer from a COM method then that sure spells trouble. A handle or pointer cannot be serialized, it is only valid in the process that created it. A custom proxy/stub is required that substitutes that handle with the actual bitmap data, transmits that data to the stub so the bitmap can to be recreated with a new handle/pointer in 64-bit process.

The odds that the COM component author took care of that are small, it isn't easy to do and he surely didn't see the requirement at the time since he assumed the component would always be used in-process. It is not anymore.

There's no simple fix for this, you have to create a proper proxy/stub and that requires having at least access to the original IDL for the component. Usually only the COM author has access to that. Bad news, I'm sure.

A possible workaround is to leave it up to .NET to take care of it. You'll need a littler helper process that runs in 32-bit mode so can load the COM component without trouble. Talk to it with one of the .NET interop mechanisms, like WCF or Remoting. Getting the bitmap serialized is now entirely in your hands.