plotting serial data using arduino and pyserial

Question

i'm trying to plot serial data of the accelerometer using arduino and pyserial,numpy and matplotlib. The problem is whenever the GUI opens, the incoming data rate becomes very slow and the plot is also very slow and if I don't open the GUI and simply print the data on the command window, the received data is fast. Plz do help out!!

Here's my code for python:

import serial
import matplotlib.pyplot as plt
import numpy as np
import time


ser = serial.Serial('COM3', 9600, timeout=0) #sets up serial connection (make sure baud          rate is correct - matches Arduino)

ser.flushInput()
ser.flushOutput()

plt.ion()                    #sets plot to animation mode

length = 500       #determines length of data taking session (in data points)
x = [0]*length               #create empty variable of length of test
y = 0
z = 0
fig, ax = plt.subplots()
line, = ax.plot(np.random.randn(100))
plt.show(block=False)
xline, = plt.plot(x)         #sets up future lines to be modified
plt.ylim(30,120)        #sets the y axis limits

#for i in range(length):     #while you are taking data
tstart = time.time()
n = 0
while time.time()-tstart < 5:
   y = (ser.readline().decode('utf-8')[:-2])
   if not (len(y)==0):
       z = float(y)
       x.append(float(z))   #add new value as int to current list

       del x[0]

       xline.set_xdata(np.arange(len(x))) #sets xdata to new list length

       xline.set_ydata(x)                 #sets ydata to new list

       #  ax.draw_artist(ax.patch)
       #  ax.draw_artist(line)
       #  fig.canvas.update()
       fig.canvas.draw()
       fig.canvas.flush_events()   
       #plt.pause(0.0001)                   #in seconds                                      #draws new plot
       #plt.show()
       n +=1
       print (z)
       print(n)




 ser.close() #closes serial connection (very important to do this! if you have an error   partway through the code, type this into the cmd line to close the connection)

Answer 1

there are several things that could speed up the process. We had the same problems, when we tried to read out geophones using an arduino.

the main trick was to buffer ten or more values before plotting and not to plot for every single new event.

Using the animation package instead of manually drawing might also speed up the process.

Have a look on our github code: https://gist.github.com/ibab/10011590

An why do you cast z 2 times to float:

z = float(y)
x.append(float(z))

Answer 2

In my experience, redrawing one line with 10 000 points takes maybe 5 milliseconds, but this depends on the backend and computer. If you are trying to do it much faster, then you are in trouble. On the other hand, updating the graph more than, say, 50 times per second is not needed or reasonable due to perception delay.

So, as MaxNoe says, you should buffer the data, if it is fast data. You may either use a fixed buffer, or a buffer with timeout. (It seems that you are still receiving a steady stream of data, otherwise you'd have problems with the ser.readline without a timeout.)

There are a few other things you might do to speed up your code, as well. Without knowing the amount of data, I am unable to say whether these have any real performance impact or not. Profiling is the key (as James Mills says).

First thing is that you should have your x and y as ndarray (NumPy array) to avoid the overhead of converting lists to arrays. Basically:

# initializing the array
x = np.zeros(length)

# adding one item to the array:
# shift the existing items and add an item to the end
x[:-1] = x[1:]
x[-1] = float(z)

Also, note that there is no need to set_xdata every round, as the xdata remains constant. Just do it once before the loop.

If you then end up with buffering, the time-based buffering goes approximately this way for each refresh cycle:

databuf = []
# collect data until you have something and at least 20 ms has gone
t0 = time.time()
while time.time() - t0 < 0.02 or len(databuf) == 0:
    ln = ser.readline()
    try:
        databuf.append(float(ln[:-2]))
    except:
        # something went wrong, but now we don't care
        pass

n_newpoints = len(databuf)
x[:-n_newpoints] = x[n_newpoints:]
x[-n_newpoints:] = databuf

# draw etc ...

Here databuf is a list, but as it is a short list, the conversion overhead is insignificant.

With these improvements you should be able to fluently update a graph of 10000 points without your computer grinding to a halt. If you are doing this on a machine with very modest performance (RaspberryPi, etc.), then the recipe is to reduce the update frequency.

Answer 3

I had a similar problem of speed when plotting data from a serial port of arduino. The solution is described in the following addresss: http://asaf.pm/wordpress/?p=113#comment-273 In this case I achieve about 700fps plotting an array with 50 points of data.

Answer 4

I think the primary reason why you face these performance issues, is that matplotlib is not primarily intended for live data. It really shines in creating excellent charts of data from a script, but adding a single sample and redrawing might not be an area where matplotlib shines.

Please consider other options for live viewing of data, such as (shameless plug) lognplot: https://github.com/windelbouwman/lognplot

Another helpful list of programs for live data viewing can be found here: https://arduino.stackexchange.com/questions/1180/serial-data-plotting-programs