从C ++更新Texture2D像素

Question

In the project, I'm using Unity3D's C# scripts and C++ via DllImports. My goal is that the game scene has 2 cubes (Cube & Cube2) which one texture of cube show the live video through my laptop camera and Unity's webCamTexture.Play() and the other one texture of cube show the processed video by external C++ function ProcessImage() .

Code Context:

In c++, I define that

struct Color32
{
    unsigned char r;
    unsigned char g;
    unsigned char b;
    unsigned char a;
};

and the function is

extern "C"
{
    Color32* ProcessImage(Color32* raw, int width, int height);
}
...

Color32* ProcessImage(Color32* raw, int width, int height)
{
    for(int i=0; i<width*height ;i++)
    {
       raw[i].r = raw[i].r-2;
       raw[i].g = raw[i].g-2;
       raw[i].b = raw[i].b-2;
       raw[i].a = raw[i].a-2;
    }
    return raw;
}

C#:

Declare and importing

public GameObject cube; 
public GameObject cube2;
private Texture2D tx2D;
private WebCamTexture webCamTexture;

[DllImport("test22")]   /*the name of Plugin is test22*/
private static extern Color32[] ProcessImage(Color32[] rawImg, 
                                                 int width, int height);

Get camera situation and set cube1, cube2 Texture

void Start()
{
    WebCamDevice[] wcd = WebCamTexture.devices;

    if(wcd.Length==0)
    {
        print("Cannot find a camera");
        Application.Quit();
    }
    else
    {
        webCamTexture = new WebCamTexture(wcd[0].name);
        cube.GetComponent<Renderer>().material.mainTexture = webCamTexture;

        tx2D = new Texture2D(webCamTexture.width, webCamTexture.height);
        cube2.GetComponent<Renderer>().material.mainTexture = tx2D;

        webCamTexture.Play();
    }
}

Send data to external C++ function by DllImports and receive processed data using Color32[] a . Finally, I'm using Unity's SetPixels32 to setup tx2D (Cube2) texture:

void Update()
{
    Color32[] rawImg = webCamTexture.GetPixels32();
    System.Array.Reverse(rawImg);

    Debug.Log("Test1");
    Color32[] a = ProcessImage(rawImg, webCamTexture.width, webCamTexture.height);
    Debug.Log("Test2");

    tx2D.SetPixels32(a);
    tx2D.Apply();
}

Results:

The result is just the texture of cube 1 show the live video and fail to show processed data using the texture of cube 2.

Error:

SetPixels32 called with invalid number of pixels in the array UnityEngine.Texture2D:SetPixels32(Color32[]) Webcam:Update() (at Assets/Scripts/Webcam.cs:45)

I don't understand that why the invalid number of pixels in the array when I input array a to SetPixels32

Any ideas?

---

UPDATE (10 Oct. 2018)

Thanks to @Programmer, now it can work by pin memory.

Btw, I find some little problem which is about Unity Engine. When the Unity Camera run between 0 to 1 second, webCamTexture.width or webCamTexture.height always return 16x16 size even requested bigger image such as 1280x720 and then it will return correct size after 1 second. (Possibly several frames) So, I reference this post and delay 2 seconds to run Process() in Update() function and reset the Texture2D size in Process() function. It will work fine:

delaytime = 0;
void Update()
{
    delaytime = delaytime + Time.deltaTime;
    Debug.Log(webCamTexture.width);
    Debug.Log(webCamTexture.height);

    if (delaytime >= 2f)
        Process();
}
unsafe void Process()
{
    ...

    if ((Test.width != webCamTexture.width) || Test.height != webCamTexture.height)
    {
       Test = new Texture2D(webCamTexture.width, webCamTexture.height, TextureFormat.ARGB32, false, false);
       cube2.GetComponent<Renderer>().material.mainTexture = Test;
       Debug.Log("Fixed Texture dimension");
    }
    ...
}

Answer 1

C# doesn't return understand the Color32 object you returned from C++. This would have worked if you made the return type to be unsigned char* then use Marshal.Copy on the C# side to copy the returned data into byte array then use Texture2D.LoadRawTextureData to load the array into your Texture2D . Here is an example that returns byte array from C#.

---

I wouldn't suggest you return data as that's costly. Fill the array you are passing to the function then simply re-assign that array to your Texture2D .

C++:

extern "C"
{
    void ProcessImage(unsigned char* raw, int width, int height);
}
...

void ProcessImage(unsigned char* raw, int width, int height)
{
    for(int y=0; y < height; y++)
    {
        for(int x=0; x < width; x++)
        {
            unsigned char* pixel = raw + (y * width * 4 + x * 4);
            pixel[0] = pixel[0]-(unsigned char)2; //R
            pixel[1] = pixel[1]-(unsigned char)2; //G
            pixel[2] = pixel[2]-(unsigned char)2; //B
            pixel[3] = pixel[3]-(unsigned char)2; //Alpha
        }
    }
}

C#:

[DllImport("test22")]
private static extern void ProcessImage(IntPtr texData, int width, int height);

unsafe void ProcessImage(Texture2D texData)
{
    Color32[] texDataColor = texData.GetPixels32();
    System.Array.Reverse(texDataColor);

    //Pin Memory
    fixed (Color32* p = texDataColor)
    {
        ProcessImage((IntPtr)p, texData.width, texData.height);
    }
    //Update the Texture2D with array updated in C++
    texData.SetPixels32(texDataColor);
    texData.Apply();
}

To use:

public Texture2D tex;

void Update()
{
    ProcessImage(tex);
}

If you don't want to use the unsafe and fixed keywords, you can also use GCHandle.Alloc to pin the array before sending it to the C++ side with GCHandle.AddrOfPinnedObject . See this post for how to do that.

---

If you are getting the following exception:

SetPixels32 called with invalid number of pixels in the array UnityEngine.Texture2D:SetPixels32(Color32[]) Webcam:Update() (at Assets/Scripts/Webcam.cs:45)

This means that the Texture2D you're calling SetPixels32 on has different size or texture dimension with the WebCamTexture texture. To fix this, before calling SetPixels32 , check if the texture dimension changed then resize your target texture to match with the WebCamTexture .

Replace

tx2D.SetPixels32(rawImg);
tx2D.Apply();

with

//Fix texture dimension if it doesn't match with the web cam texture size
if ((tx2D.width != webCamTexture.width) || tx2D.height != webCamTexture.height)
{
    tx2D = new Texture2D(webCamTexture.width, webCamTexture.height, TextureFormat.RGBA32, false, false);
    Debug.Log("Fixed Texture dimension");
}

tx2D.SetPixels32(rawImg);
tx2D.Apply();