Mailboxprocessor, latest message

Question

The setup is similar to this . One agent, ( dataSource ) is generating data, and a single agent ( dataProcessor ) is processing the data. There is a lot more data being generated than dataProcessor can process, and I am not interested in processing all messages, just processing the latest piece of data.

One possible solution, proposed there by Jon Harrop there "is to greedily eat all messages in the inbox when one arrives and discard all but the most recent". Another approach is not to listen for all messages, but rather for dataProcessor to PostAndReply dataSource to get the latest piece of data.

What are the pros and cons of these approaches?

Answer 1

This is an intriguing question and there are quite likely several possible perspectives. I think the most notable aspect is that the choice will affect how you design the API at the interface between the two components:

1. In "Consume all" approach, the producer has a very simple API where it triggers some event whenever a value is produced and your consumer will subscribe to it. This means that you could have other subscribers listening to updates from the producer and doing something else than your consumer from this question.

2. In "Call to get latest" approach, the producer will presumably need to be written so that it keeps the current state and discards old values. It will then provide blocking async API to get the latest value. It could still expose an event for other consumers though. The consumer will need to actively poll for changes (in a busy loop of some sorts).

3. You could also have a producer with an event as in "Consume all", but then create another component that listens to any given event, keeps the latest value and makes it available via a blocking async call to any other client.

Here some advantages/disadvantages I can think of:

• In (1) the producer is very simple; the consumer is harder to write
• In (2) the producer needs to do a bit more work, but the consumer is simple
• In (3), you are adding another layer, but in a fairly reusable way.

I would probably go with either (2) (if I only need this for one data source) or with (3) after checking that it does not affect the performance.

As for (3), the sketch of what I was thinking would look something like this:

type KeepLastMessage<'T> = 
  | Update of 'T
  | Get of AsyncReplyChannel<'T>

type KeepLast<'T>(initial:'T, event:IObservable<'T>) = 
  let agent = MailboxProcessor.Start(fun inbox -> 
    let rec loop last = async {
      let! msg = inbox.Receive()
      match msg with 
      | Update last -> return! loop last
      | Get ch -> ch.Reply(last); return! loop last }
    loop initial)
  member x.AsyncGet() = agent.PostAndAsyncReply(Get)