Sending screen capture from a C# Unity project to a Python socket

Question

I'm trying to do a project in which I have a virtual environment with cameras and want to send screenshots of selected cameras through a socket to a python client that will be running on a raspberry pi zero. In the raspberry I will have additional processing but that is not my current issue.

I've looked into screencapturing in unity, there was a project that appeared to do the capturing part quite right although I'm having some trouble adapting the code from https://assetstore.unity.com/packages/tools/camera/screenshot-helper-102472 I have

 ScreenshortHelper.iCaptureWithCamera(camera, (texture2D) => {//Code}};

Would I be able to insert the "send" code in the "//Code" block?

Also that part for me is a bit tricky, sending the "texture" through the socket, I was thinking about sending some parameters like size and dimensions first so that I know what I would be receiving on the python end. Does this make sense?

I appreciate any help that you can give me!

Thanks in advance.

Answer 1

First, you need to configure the Transport Layer API in your unity code, and put references to your hostId and your channelId somewhere where that function can access it.

Then, that anonymous function there can be:

(texture2D) => {
    // Encode it to PNG bytes and put it into a buffer:
    byte[] buffer = texture2D.EncodeToPNG();

    // Connect to the python listener:
    byte error;
    connectionId = NetworkTransport.Connect(hostId, "192.168.1.42", 8888, 0, out error);

    // Send:
    int bufferLength = bytes.Length;
    NetworkTransport.Send(hostId, connectionId, channelId, buffer, bufferLength, out error);

    // Disconnect:
    NetworkTransport.Disconnect(hostId, connectionId, out error);
}

And then on the other side, you can use Python to listen on a binary stream, and then do whatever you need to do with it. There is some useful information on how to read an image over a socket in python here: Receive Image using socket programming in Python