读取串行命令会花费太多时间

Question

so following problem: I am currently writing a small program for my NUCLEO F207ZG which provides an interface for other services over the serial port. The goal of my program is to expose data with commands like ?variable and set values with !variable <value> (eg. ?threshold returns 1400 and !threshold 1234 sets threshold to 1234). Additonally the variables in my program are synced with the EEPROM to persist the data.

For this whole process I've been given a codebase which includes a SerialCommands.h library.

Problem

The main problem is that setting values takes up way too much time and breaks the serial buffer (or something similiar) if other commands are executed immediatly after setting. I've written a few unit tests in python (using pyserial ) that execute get requests (such as ?threshold ) flawlessy, as often as possible. However, If I execute a set command (such as !threshold 1400 ) I need to wait AT LEAST four seconds before making any other requests, otherwise the serial interface/buffer seems to lose some data. The same thing happens if I try to make any set -> get requests on the Arduino Serial Monitor. Here is a short example:

1. ?threshold => returns '1400'
2. !threshold 1234
3. ?threshold => nothing happens
4. ?threshold => nothing happens
5. ?threshold => returns 'unrecognized command ???threshold ' (see my code for this functionality)

EDIT: Something important which I forgot to mention. After the program has recovered (after step 5) the value can be queried correctly.

6. ?threshold => returns '1234'

I also have a blinking status LED (in 500ms steps) and if I set something the blinking stops noticeably for around a second.

Code

So here is (non-working) simplified example of my codebase:

#include <SimpleTimer.h>    // https://github.com/marcelloromani/Arduino-SimpleTimer
#include <SerialCommands.h> // https://github.com/ppedro74/Arduino-SerialCommands/
#include <EEPROM.h>
#include "EEPROMAnything.h"
char serial_command_buffer_[64];

SerialCommands serial_commands_(&Serial, serial_command_buffer_, sizeof(serial_command_buffer_), "\r\n", " ");

SimpleTimer timer;
int filter_threshold;

void cmd_unrecognized(SerialCommands* sender, const char* cmd) {
  sender->GetSerial()->print("Unrecognized command [");
  sender->GetSerial()->print(cmd);
  sender->GetSerial()->println("]");
}

void set_arg(SerialCommands* sender, int& injectedVar) {
  char* temp = sender->Next();
  injectedVar = atoi(temp);
}

void set_arg(SerialCommands* sender, int& injectedVar, int address) {
  set_arg(sender, injectedVar);
  EEPROM_writeAnything(address, injectedVar);
}

void echo(SerialCommands* sender, int var) { sender->GetSerial()->println(var); }
void echo(SerialCommands* sender, String var) { sender->GetSerial()->println(var); }

void get_pressure_threshold(SerialCommands* sender) { echo(sender, filter_threshold); } //?threshold
void set_pressure_threshold(SerialCommands* sender) { set_arg(sender, filter_threshold, ADDR_THRESHOLD); } //!threshold <int>

void main_timer() {
  mock_changes();
}

SerialCommand cmd_getpressthreshold("?threshold", get_pressure_threshold);
SerialCommand cmd_setpressurethreshold("!threshold", set_pressure_threshold);

void add_serial_commands() {
  serial_commands_.SetDefaultHandler(cmd_unrecognized);
  serial_commands_.AddCommand(&cmd_getpressthreshold);
  serial_commands_.AddCommand(&cmd_setpressurethreshold);
}

void setup() {
  pinMode(PB0, OUTPUT);
  pinMode(PB7, OUTPUT);
  pinMode(PB14, OUTPUT);

  Serial.begin(9600);

  timer.setInterval(500, main_timer);
  add_serial_commands();
}

void loop() {
  serial_commands_.ReadSerial();
  timer.run();
}

I've mostly omitted the code covering the EEPROM functionality as I've confirmed that it is not the source of the slow/unpredictable behaviour.

Debugging attempts

So the main problem seems the be concerning the time of the setting process, as it works if I do it slow enough. I've tried timing all the parts of the codebase and I simply can't explain the delay when setting anything and the required delay for future requests. Setting the EEPROM values takes at most 75 ms, ReadSerial() is so fast I can barely measure it in 10^-3 seconds.

At this point I am pretty sure that I am doing something wrong with reading/writing the serial buffer/interface. I've tried flush() ing it after every set call which does nothing.

I've also thought that maybe my PCs serial interface (usb port) is somehow causing this problem as this would explain why no part of the codebase takes up much time. However, this does not really fit with the fact that the loop() is delayed and the LED stops blinking.

One of the strangest things for me about this whole problem is how the serial interface receives the cmds after spamming them a few times. The cmd_unrecognized(...) function does not receive ?threshold?threshold?threshold but ???threshold , which makes no sense to me.

I know this post has been quite long but I hope I've provided a relatively clear picture of my problem and I hope any of you guys have got an idea how to fix this time-consuming mess of a problem.

EDIT2: After playing around with in the SerialCommands.h debug mode I've gotten the following output when quickly entering !pidp 10 and then ?pidp :

Read: bufLen=63 bufPos=0 termPos=0 ch=[!]
Read: bufLen=63 bufPos=1 termPos=0 ch=[p]
Read: bufLen=63 bufPos=2 termPos=0 ch=[i]
Read: bufLen=63 bufPos=3 termPos=0 ch=[d]
Read: bufLen=63 bufPos=4 termPos=0 ch=[p]
Read: bufLen=63 bufPos=5 termPos=0 ch=[ ]
Read: bufLen=63 bufPos=6 termPos=0 ch=[1]
Read: bufLen=63 bufPos=7 termPos=0 ch=[0]
Read: bufLen=63 bufPos=8 termPos=0 ch=#13
Read: bufLen=63 bufPos=9 termPos=1 ch=#10
Received: [!pidp 10]
Matched #30
Read: bufLen=63 bufPos=0 termPos=0 ch=[?]

This somehow looks like the full ?pidp didn't fit the buffer, however, there is a buffer full message and it is not shown.

Answer 1

It sounds like it could be caused by your Python program. Did you remember to add a null-terminator ( \\0 ) after the 1234 integer? Serial commands generally require strings, which must end with a null-terminator. If not your Arduino might keep on waiting for the terminator, and only stops when a timeout occurs or watchdog causes a reset (which could be 4 seconds). That would explain the hold-up.

Answer 2

Ultimatively I've not really found out/understood the problem with the serial buffer but I've managed to implement a basic workaround. Instead of always writing to the EEPROM (and therefore taking up to 75ms), I only set the serial input to the respective variables (which takes at most 3ms). In order to persist changes I've added a !commit command which saves all values. With this change I've centralized the issue of messing with the serial interface if my program takes too long, therefore, I will have to handle this in the instance that is calling the serial interface.

To summarize the problem to my knowledge: If my code is blocking/working while any commands are sent to the serial interface, the serial buffer is filled in a strange, non-predictable way (instead of queuing the buffer input as expected). At the end of the day I just have to avoid this case. This is definitely not perfect, but it works. Thanks for the help to anyone.