Viewing data in a circular buffer in real-time

Question

I have an incoming stream of messages, and want a window that allows the user to scroll through the messages.

This is my current thinking:

• Incoming messages go into single producer single consumer queue
• A thread reads them out and places them into a circular buffer with a sequential id
• This way I could have multiple incoming streams safely placed in the circular buffer and it decouples the input
• Mutex to coordinate circular buffer access between the UI and the thread
• Two notifications from thread to UI one for the first id and one for the last id in the buffer when ever either changes.
• This allows the UI to figure out what it can display, which parts of the circular buffer it needs access, delete overwritten messages. It only accesses the messages required to fill the window at its current size and scroll position.

I'm not happy about the notification into the UI. It would be generated with high frequency. These could be queued or otherwise throttled; latency should not affect the first id but delays in handling the last id could cause problems in corner cases such as viewing the very end of a full buffer unless the UI makes a copy of the messages it displays, which I would like to avoid.

Does this sound like the right approach? Any tweaks that could make it a bit more palatable?

Answer 1

(See Effo EDIT below, and this part is deprecated) The ring buffer is not necessary if there's a queue between the thread and each UI.

When message arrived, the thread pop it and push it to a UI's queue accordingly.

Furthermore each UI.Q could be operated atomically too. There's no mutex needed. Another benefit is that each message only had been copied twice: one is to low level queue, anther is to the display, because storing the message to elsewhere is not necessary (just assign a pointer from low level queue to UI.Q should be enough if C/C++).

So far the only concern is that might the length of a UI.Q is not run-time enough when messaging traffic is heavy. Per this question, you can either use a dynamic-length queue or let the UI itself stores the overflowed message to a posix memory-mapped file. High efficiency if using posix mapping, even though you are using a file and need to do extra message copying. But anyway it is only the exception handling. Queue could be set to a proper size so that normally you'll get excellent performances. The point is that when the UI need to store overflowed message to a mapped file, it should perform highly-concurrent operation too so that it will not affect the low level queue.

I prefer to dynamic-size queue proposal. It seems we have lots of memory on modern PCs.

see the document EffoNetMsg.pdf at http://code.google.com/p/effonetmsg/downloads/list to learn more about lock-free, queue facilities and Highly-concurrent Programming Models.

---

Effo EDIT@2009oct23: Show a Staged Model which support random message accessing for scrolling of message viewers.

                         +---------------+ 
                     +---> Ring Buffer-1 <---+
                     |   +---------------+   |
                  +--+                       +-----+
                  |  |   +---------------+   |     |
                  |  +---> Ring Buffer-2 <---+     |
                  |      +---------------+         |
                  |                                |
          +-------+-------+            +-----------+----------+
          |   Push Msg &  |            |   GetHeadTail()      |
          |  Send AckReq  |            |  & Send UpdateReq    |
          +---------------+            +----------------------+
          |App.MsgStage() |            |   App.DisPlayStage() |
          +-------+-------+            +-----------+----------+
                  | Pop()                          | Pop()         
 ^              +-V-+                            +-V-+ 
 | Events       | Q |    Msg Stage |             | Q |  Display Stage
 | Go Up        | 0 |   Logic-Half |             | 1 |   Logic-Half      
-+------------- |   | -------------+------------ |   | ---------------
 | Requests     |   |    I/O-Half  |             |   |    I/O-Half
 | Move Down    +-^-+              |             +-^-+   
 V                | Push()                         |     
   +--------------+-------------+                  |
   |   Push OnRecv Event,       |          +-------+-------+
   | 1 Event per message        |          |               | Push()
   |                            |   +------+------+ +------+------+
   |  Epoll I/O thread for      |   |Push OnTimer | |Push OnTimer |
   |multi-messaging connections |   |  Event/UI-1 | |  Event/UI-2 |
   +------^-------^--------^----+   +------+------+ +------+------+
          |       |        |               |               |                   
Incoming msg1    msg2     msg3        Msg Viewer-1    Msg Viewer-2              

The Points:

1 You understand different Highly-concurrent Models, specific shown in above figure, a Staged Model; so that you'll know why it runs fast.

2 Two kinds of I/O, one is Messaging or Epoll Thread if C/C++ and GNU Linux 2.6x; another is Displaying such as drawing screen or printing text, and so on. the 2 kinds of I/O are processed as 2 Stages accordingly. Note if Win/MSVC, use Completion Port instead of Epoll.

3 Still 2 message-copyings as mentioned before. a) Push-OnRecv generates the message ("CMsg *pMsg = CreateMsg(msg)" if C/C++); b) UI read and copy message from it's ring buffer accordingly, and only need to copy updated message parts, not the whole buffer. Note queues and ring buffers are only store a message handle ("queue.push(pMsg)" or "RingBuff.push(pMsg)" if C/C++), and any aged-out message will be deleted ("pMsg->Destroy()" if C/C++). In general the MsgStage() would rebuild the Msg Header before push it into the ring buffer.

4 After an OnTimer event, the UI will receive update from upper layer which contains new Head/Tail indicators of the ring buff. so UI could update the display accordingly. Hope UI has a local msg buffer, so don't need to copy the whole ring buffer, but just update. see point 3 above. If need to perform random-accessing on ring buffer, you could just let UI generate OnScroll event. actually if UI has a local buffer, OnScroll might be not necessary. anyway, you can do it. Note UI will determine wheter to discard an aged-out message or not, say generate OnAgedOut event, so that the ring buffers could be operated correctly and safely.

5 Exactly, OnTimer or OnRecv is the Event name, and OnTimer(){} or OnRecv(){} would be executed in DisplayStage() or MsgStage(). Again, Events go upwards and Requests go downstream, and this might be different from that what you had though or seen before.

6 Q0 and 2 ring buffers could be implemented as lock-free facilities to improve performances, since single producer and single consumer; no lock/mutex needed. while Q1 is somthing different. But I believe you are able to make it single producer and single consumer too by changing the above design figure slightly, eg add Q2 so every UI has a queue, and DisplayStage() could just polling Q1 and Q2 to process all events correctly. Note Q0 and Q1 are Event-Queue, the Request-Queues are not shown in above figure.

7 MsgStage() and DisplayStage() are in a single StagedModel.Stage() sequentially, say the Main Thread. Epoll I/O or Messaging is another thread, the MsgIO Thread, and every UI has an I/O thread, say Display Thread. so in above figure, there're 4 threads in total which are running concurrently. Effo had tested that just one MsgIO Thread should be enough for multi-liseners plus thousands of messaging clients.

Again, see the document EffoNetMsg.pdf at http://code.google.com/p/effonetmsg/downloads/list or EffoAddons.pdf at http://code.google.com/p/effoaddon/downloads/list to learn more about Highly-concurrent Programming Models and network messaging; see EffoDesign_LockFree.pdf at http://code.google.com/p/effocore/downloads/list to learn more about lock-free facilities such as lock-free queue and lock-free ring buffer.

Answer 2

The notification to the GUI shouldn't contain the ID, ie the current value. Instead it should just say "the current value has changed", and then let the GUI read the value: because there may be a delay between sending the notification and the GUI reading the value, and you want the GUI to read the current value (and not a potentially stale value). You want it to be an asynchronous notification.

Also you can afford to throttle notifications, eg send no more than 5 or 20 per second (delay a notification by up to 50 to 200 msec if necessary).

Also the GUI will inevitably be making a copy of the message it displays, in the sense that there will be a copy of the message on the screen (in the display driver)! As for whether the GUI makes a copy into a private RAM bufferof its own, although you might not want to copy the whole message, you might find it safer/easier to have a design where you copy just as much of the message as you need to paint/repaint the display (and because you can't paint very much on a screen at one time, that implies that the quantity of data which you need to copy to do that would be trivial).