Issues with TCP on Arduino. Reading Strings from a Python TCP Socket Server

Question

I am experimenting with my new arduino UNO Rev 3 and a simple python socket server. The server code is:

##server.py
from socket import *      #import the socket library

#MY FUNCTIONS
def send(data = str):
    conn.send(data.encode())
def recieve():
    recieve.data = conn.recv(BUFSIZE)
    recieve.data = recieve.data.decode()
    return recieve.data

##let's set up some constants
HOST = ''    #we are the host
PORT = 29876    #arbitrary port not currently in use
ADDR = (HOST,PORT)    #we need a tuple for the address
BUFSIZE = 4096    #reasonably sized buffer for data

## now we create a new socket object (serv)
## see the python docs for more information on the socket types/flags
serv = socket( AF_INET,SOCK_STREAM)

##bind our socket to the address
serv.bind((ADDR))    #the double parens are to create a tuple with one element
serv.listen(5)    #5 is the maximum number of queued connections we'll allow
print(" ")
print("Listening")
print(" ")
conn,addr = serv.accept() #accept the connection
print("Connection Established")
send('HELLOCLIENT')
send('%')
leave()

All I'm trying to do is confirm I can communicate with the Arduino and build from there. Here is the relevant Arduino code:

void loop()
{
    int n =-1;
    while (client.connected())
    {
        n++;
        char recieved = client.read();
        inData[n] = recieved; 

        // Process message when new line character is recieved
        if (recieved == '%')
        {
            Serial.print("Arduino Received: ");
            Serial.print(inData);
            n = -1;
        }
    }
    Serial.println();
    Serial.println("Disconnecting.");
    client.stop();

}

I keep getting this output:

Connected
Arduino Received: ÿÿÿÿÿÿÿÿÿÿHELLOCLIENT%
disconnecting.

Why am I receiving all those padded characters? I've researched different encoding methods and tried ASCII and UTF-32 but it keeps doing it, what am I missing?

Answer 1

I'm willing to bet this is your problem:

Whatever your client object is, it's probably either an Arduino Ethernet Client , or one of the many other types with the same interface.

If you look at the docs for the read method, it returns:

The next byte (or character), or -1 if none is available.

So, when you first connect, you immediately start reading, but nothing has been sent yet, so the first few reads all return -1.

Since this is a char , -1 and 255 are the same value. And the character 255, in Latin-1 (and Latin-1-derived character sets) is ÿ , so that's why you're seeing a bunch of ÿ at the start. Changing the encoding will either give you encoding errors or interpret the 255 as some different character, it won't get rid of it.

So, how do you fix this?

---

Some client classes (although not all, and the basic Ethernet Client is not one of them) can be put into blocking mode, which means read will wait until there's something to read.

---

The Client protocol doesn't actually specify what type read returns. Some Client types return int . And in an int , unlike a char , you can distinguish -1 and 255 . But once you cast it to char , you throw away the other 3 bytes that and can no longer do so. So, depending on the type you're using, you may be able to do this:

int recieved = client.read();
if (recieved == -1) continue;
inData[n] = (char)recieved;

---

If you only want to send ASCII text (so only values up to 127 are valid), you know that the -1 will never be an actual character. In that case, you can just do this:

char recieved = client.read();
if (recieved == -1) continue;

---

Most client types have a method available , which tells you how many bytes are available to read. If this returns 0 , wait a bit and try again instead of reading:

if (!client.available()) {
    delay(200);
    continue;
}

Many Arduino samples put this together with a delay(500) or longer at the start of the program, because there's no point wasting time checking for data when the odds are very high that nothing is there yet.

---

You can add a "start" character to your protocol, just like the "end" character. The same way you keep reading until you get a % and then disconnect, you can first keep reading until you get, say, a $ character, throwing away everything up to that point:

while (client.connected()) {
    char received = client.read();
    if (received == '$') break;
}
while (client.connected()) {
    /* your original code here */
}