having trouble with sockets in unity3d

Question

Made a Hololens application that communicates with a python script. The python script sends PLC(Siemens S7-1200) variables to the hololens in the form of JSON. The C# script processes the JSON. While the script reads data from the plc, It's also possible to control the variables with the sendMessage method inside the C# script. I've made a seperate python script that controls the plc, this I won't share.

my Unity3d C# script:

using UnityEngine;
using System;
using System.IO;
using System.Text;
using System.Linq;
using HoloToolkit.Unity;
using System.Collections.Generic;
using UnityEngine.UI;
using Newtonsoft.Json;
#if !UNITY_EDITOR
using Windows.Networking.Sockets;
using Windows.Networking.Connectivity;
using Windows.Networking;

#endif
#if !UNITY_EDITOR
public class RootObject
{
    public int Index { get; set; }
    public bool Moto { get; set; }
    public bool Start { get; set; }
    public bool StartWINCC { get; set; }
    public bool Stop { get; set; }
    public bool StopWINCC { get; set; }
    public bool Tag1 { get; set; }
    public bool Tag2 { get; set; }
}
#endif
public class UDPCommunication : Singleton<UDPCommunication>
{
    // Connection variables
    public string port = "8000";
    public string externalIP = "172.16.24.136";
    public string externalPort = "8000";
    // UI/Text elements
    public Text testert;
    public Image moto;
    public Image start;
    public Image startwincc;
    public Image stop;
    public Image stopwincc;
    public Image tag1;
    public Image tag2;
    public String uitext;
    // Sets up a Queue
    public readonly static Queue<Action> ExecuteOnMainThread = new Queue<Action>();


#if !UNITY_EDITOR
    // Socket initialization
    DatagramSocket socket;
#endif
#if !UNITY_EDITOR
    // use this for initialization
    async void Start()
    {
        Debug.Log("Waiting for a connection...");

        socket = new DatagramSocket();
        socket.MessageReceived += Socket_MessageReceived;

        HostName IP = null;
        try
        {
            var icp = NetworkInformation.GetInternetConnectionProfile();

            IP = Windows.Networking.Connectivity.NetworkInformation.GetHostNames()
            .SingleOrDefault(
                hn =>
                    hn.IPInformation?.NetworkAdapter != null && hn.IPInformation.NetworkAdapter.NetworkAdapterId
                    == icp.NetworkAdapter.NetworkAdapterId);

            await socket.BindEndpointAsync(IP, port);
        }
        catch (Exception e)
        {
            Debug.Log(e.ToString());
            Debug.Log(SocketError.GetStatus(e.HResult).ToString());
            return;
        }

        var message = "hello from " + IP;
        await SendMessage(message);
        await SendMessage("hello");

        Debug.Log("exit start");
    }

    private async System.Threading.Tasks.Task SendMessage(string message)
    {
        using (var stream = await socket.GetOutputStreamAsync(new Windows.Networking.HostName(externalIP), externalPort))
        {
            using (var writer = new Windows.Storage.Streams.DataWriter(stream))
            {
                var data = Encoding.UTF8.GetBytes(message);

                writer.WriteBytes(data);
                await writer.StoreAsync();
                Debug.Log("Sent: " + message);
            }
        }
    }
#else
    // Use this for initialization.
    void Start()
    {

    }
#endif
    // Update is called once per frame.
    void Update()
    {
        // Dequeues items until there are no more items on the queue.
        while (ExecuteOnMainThread.Count > 0)
        {
            ExecuteOnMainThread.Dequeue().Invoke();
        }
    }
#if !UNITY_EDITOR
    // this method is purely for setting the UI elements based on the received JSON string.
    private void setStuff(string input){
        // Turns the json string into an object
        var results = JsonConvert.DeserializeObject<RootObject>(input);
        // Sets the UI element(and converts it to string, because it is an int)
        testert.text = results.Index.ToString();
        // sets the image green if the variable is true, and red if it's not
        if (results.Moto == true)
        {
            moto.GetComponent<Image>().color = Color.green;
        }
        else
        {
            moto.GetComponent<Image>().color = Color.red;
        }
        // sets the image green if the variable is true, and red if it's not
        if (results.Start == true)
        {
            start.GetComponent<Image>().color = Color.green;
        }
        else
        {
            start.GetComponent<Image>().color = Color.red;
        }
        // sets the image green if the variable is true, and red if it's not
        if (results.StartWINCC == true)
        {
            startwincc.GetComponent<Image>().color = Color.green;
        }
        else
        {
            startwincc.GetComponent<Image>().color = Color.red;
        }
        // sets the image green if the variable is true, and red if it's not
        if (results.Stop == true)
        {
            stop.GetComponent<Image>().color = Color.green;
        }
        else
        {
            stop.GetComponent<Image>().color = Color.red;
        }
        // sets the image green if the variable is true, and red if it's not
        if (results.StopWINCC == true)
        {
            stopwincc.GetComponent<Image>().color = Color.green;
        }
        else
        {
            stopwincc.GetComponent<Image>().color = Color.red;
        }
        // sets the image green if the variable is true, and red if it's not
        if (results.Tag1 == true)
        {
            tag1.GetComponent<Image>().color = Color.green;
        }
        else
        {
            tag1.GetComponent<Image>().color = Color.red;
        }
        // sets the image green if the variable is true, and red if it's not
        if (results.Tag2 == true)
        {
            tag2.GetComponent<Image>().color = Color.green;
        }
        else
        {
            tag2.GetComponent<Image>().color = Color.red;
        }
    }
    //this method gets called when a message is received
    private async void Socket_MessageReceived(Windows.Networking.Sockets.DatagramSocket sender, Windows.Networking.Sockets.DatagramSocketMessageReceivedEventArgs args)
    {
        // Read the received message.
        Stream streamIn = args.GetDataStream().AsStreamForRead();
        StreamReader reader = new StreamReader(streamIn);
        string message = await reader.ReadLineAsync();
        Debug.Log("MESSAGE: " + message);
        // if the count is zero, the message will be relayed to the setStuff method, which processes the string continuously.
        // The message contains a JSON string which is received from the server.
        if (ExecuteOnMainThread.Count == 0)
        {
            ExecuteOnMainThread.Enqueue(() =>
            {
                setStuff(message);
            });
        }
    }
#endif
}

And this is my python server code:

import socket,time
from read import CustOPCLib


class BroadcastServer(object):
    def __init__(self,host,port):
        self._host = host
        self._port = port
        self.sock = None
    def connect(self):
        self.sock = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
        self.sock.connect((self._host,self._port))
    def sendMsg(self,msg):
        self.sock.send(msg.encode('utf8'))
    def disconnect(self):
        self.sock.close()

if __name__ == '__main__':
    c = CustOPCLib()
    c.connect()
    Host = '172.16.24.174'
    Port = 8000
    ser = BroadcastServer(Host,Port)
    ser.connect()
    msg = 'test'
    i = 0
    while True:
        i = i + 1
        msg += (str(i))
        ser.sendMsg(c.opcjson())
        time.sleep(0.25)
    ser.disconnect()

This is the read script(the one in p.opcjson()):

from opcua import ua, Client
import json

'''This class makes connection to the OPC-UA server, and has functions that can be used to return the current values of a specific
variable'''
class ReadVariables(object):
    '''init function, contains the node locations of the OPC-UA server'''
    def __init__(self):
        self.jsonobj = {}
        self.Index = "ns=7;s=S7-1200 station_2.PLC_OPC_HOLOLENS.DATA.Index"
        self.Moto = "ns=7;s=S7-1200 station_2.PLC_OPC_HOLOLENS.MOTO"
        self.Start = "ns=7;s=S7-1200 station_2.PLC_OPC_HOLOLENS.START"
        self.StartWincc = "ns=7;s=S7-1200 station_2.PLC_OPC_HOLOLENS.STARTWINCC"
        self.Stop = "ns=7;s=S7-1200 station_2.PLC_OPC_HOLOLENS.STOP"
        self.StopWINCC = "ns=7;s=S7-1200 station_2.PLC_OPC_HOLOLENS.STOPWINCC"
        self.Tag_1 = "ns=7;s=S7-1200 station_2.PLC_OPC_HOLOLENS.Tag_1"
        self.Tag_2 = "ns=7;s=S7-1200 station_2.PLC_OPC_HOLOLENS.Tag_2"
        self.client = None

    '''This function connects to the OPC-UA server'''
    def connect(self):
        print("Connecting to OPC-UA server")
        self.client = Client("opc.tcp://Wilrik-PC:4845")
        self.client.connect()
        print("Connected to OPC-UA server") 

    '''This function can be used to disconnect from the OPC-UA server'''
    def disconnect(self):
        print("Disconnecting from OPC-UA server")
        self.client.disconnect()

    '''This function returns the root node on the OPC-server'''
    def get_root_node(self):
        root = self.client.get_root_node()
        print("Root node is:", root)
        return root

    '''This function returns the objects node on the OPC-server'''
    def get_objects_node(self):
        objects = self.client.get_objects_node()
        print("Objects node is: ",objects)
        return objects

    '''This function returns the children of the objects node on the OPC-server'''
    def get_objects_node_children(self):
        objects = self.client.get_objects_node()
        print("children of objects node is:",objects.get_children())
    ################################################# VARIABLES ###########################################################
    '''This function returns the value of the index variable on the OPC-server'''
    def index(self):
        index = self.client.get_node(self.Index)
        return index.get_value()

    '''This function returns the value of the moto variable on the OPC-server'''
    def moto(self):
        moto = self.client.get_node(self.Moto)
        return moto.get_value()

    '''This function returns the value of the start variable on the OPC-server'''
    def start(self):
        start = self.client.get_node(self.Start)
        return start.get_value()

    '''This function returns the value of the startwincc variable on the OPC-server'''
    def startwincc(self):
        startwincc = self.client.get_node(self.StartWincc)
        return startwincc.get_value()

    '''This function returns the value of the stop variable on the OPC-server'''
    def stop(self):
        stop = self.client.get_node(self.Stop)
        return stop.get_value()

    '''This function returns the value of the stopwincc variable on the OPC-server'''
    def stopwincc(self):
        stopwincc = self.client.get_node(self.StopWINCC)
        return stopwincc.get_value()

    '''This function returns the value of the Tag_1 variable on the OPC-server'''
    def tag_1(self):
        tag1 = self.client.get_node(self.Tag_1)
        return tag1.get_value()

    '''This function returns the value of the Tag_2 variable on the OPC-server'''
    def tag_2(self):
        tag2 = self.client.get_node(self.Tag_2)
        return tag2.get_value()
    ################################################# VARIABLES ###########################################################

    ################################################# JSON ################################################################
    def opcjson(self):
        self.jsonobj['Index'] = self.index()
        self.jsonobj['Moto'] = self.moto()
        self.jsonobj['Start'] = self.start()
        self.jsonobj['StartWINCC'] = self.startwincc()
        self.jsonobj['Stop'] = self.stop()
        self.jsonobj['StopWINCC'] = self.stopwincc()
        self.jsonobj['Tag1'] = self.tag_1()
        self.jsonobj['Tag2'] = self.tag_2()
        obj = json.dumps(self.jsonobj)
        return self.jsonobj
    ################################################# JSON ################################################################

UPDATE 1: Applied DoctorPangloss's suggestion, while I got it to work, it's still not working the way I want. My current situation: I start up my Unity3d application, then i start up my python application, unity receives data, I close down the python script and restart it, I receive data only once, and then hangs.

UPDATE 2: Removed the question, will leave this up for those who could use it. This doesn't include the controls script, you'll have to make that yourself.

Answer 1

Your issue is with this line:

Debug.Log("client message received as: " + clientMessage);

You can't access the Unity thread (ie, Unity objects) from another thread (ie, your listening thread). Enqueue received packages into a ConcurrentQueue queue , and dequeue from queue until it is empty every frame inside an Update function.

Your second issue is with your ConcurrentQueue implementation. It's wrong. The place you found it from had this comment:

This is not ConcurrentQueue.. but just queue + lock

https://gist.github.com/jaredjenkins/5421892

Try to adapt a real concurrent queue implementation, like this one: https://github.com/mono/mono/blob/master/mcs/class/referencesource/mscorlib/system/collections/Concurrent/ConcurrentQueue.cs . Use TryDequeue(out item) to actually dequeue, not peek or Dequeue .

Your third issue is with these lines:

using (NetworkStream stream = connectedTcpClient.GetStream())
                {
                    int length;
                    // Read incomming stream into byte arrary.                      
                    while ((length = stream.Read(bytes, 0, bytes.Length)) != 0)

TCP isn't a messaging protocol. If you want to send messages, use web sockets, or length-prefix the length of the content you're going to send from one place to another, even if it's just strings. This isn't necessarily going to bite you right now, but it will later.

Your fourth issue is with p.opcjson() . Maybe read this once, and save it to a variable? It could be blocking on the second call, unexpectedly; I'm not sure what's going on here.

Overall, I think there's probably an issue in both your C# listening code and your Python code. Writing your own TCP sockets code is extremely error-prone, unfortunately. Consider using web sockets from the Asset Store and in Python, whose semantics are significantly more intuitive.