Mosquitto QoS2 bug - or subtle spec interpretation?

Question

Can anyone explain why this MQTT QoS message never gets sent? ANSWER : set persistent true is need in conf file, then the exact expected behaviour appears.

More detail

I have a simple question (although with a complex starting condition). If the answer is "yes" then there is a bug in mosquitto if "no" then its a very subtle interpretation of the spec that I (and I suspect many others) do not fully grasp. In the second more likely case, the implication is that it is not possible to write a fully QoS2 compliant "sequence tester" on a single devce using sub and pub to the same topic.

Setup code which subscribes to T @ QoS2 and starts clean==false every time. Every 1sec publishes T with increasing integer value such that in normal course of operation the squence of events is:

• Client - > server TX 1
• <TX 4wayhandshake>
• server pubs 1
• <RX 4wayhandshake>
• RX 1
• TX 2
• <TX 4wayhandshake>
• server pubs 2
• <RX 4wayhandshake>
• RX 2

etc

I then deliberately break the connection to the server at random intervals, to test my recovery procedures.

On restart my test client correctly recognises a failed TX as it has stored state eg failed TX3 and resends, and of course mosquitto re-acks...whatever the terminology, if the break is in the middle of the TX 4-way, my code recovers correctly, if it is in the middle of the RX 4-way then mosquitto immediately "recovers" its own failed TX and everything continues exactly as expected. I never get any "missed" value of N, and I declare my code fully working. So what's my problem?

Connection break / restart behaviour anomaly Consider when the connection breaks exactly between the TX and the server pub (what would be my next RX), ie TX3 succeeds fully PUB/REC/REL/COMP and if there were no break, I would print "RX 3". Next I use mqtt-spy to publish T[666]. then I restart my client. First thing I get is the 666 message from the server, which sounds right to me and seems to agree with:

When a Server takes ownership of an incoming Application Message it MUST add it to the Session state of those clients that have matching Subscriptions. Matching rules are defined in Section 4.7 [MQTT-4.5.0-1]. and

[MQTT-3.1.2-4] | If CleanSession is set to 0, the Server MUST resume communications with the Client based on state from the current Session (as identified by the Client identifier). If there is no Session associated with the Client identifier the Server MUST create a new Session. The Client and Server MUST store the Session after the Client and Server are disconnected.

But I never see RX3 again.

Surely, the same rule is true for TX3? Server took ownership when it pubcomp'd me, and MUST have added it to the open session we shared at the time, and therefore it knows that I am a QoS2 subscriber to whom that message has not yet been sent, in the same way as 666, and therefore - by my logic - I MUST receive that RX3 re-send as well as the 666?

The only thing that makes any sense to me other than mosquitto being wrong in failing to re-send T[3] is a subtle (and confusing) interpretation the word further in:

[MQTT-3.1.2-5] | After the disconnection of a Session that had CleanSession set to 0, the Server MUST store further QoS 1 and QoS 2 messages that match any subscriptions that the client had at the time of disconnection as part of the Session state.

I see how 666 is "further" since it occurs after the break. I see also that 3 was before the break and therefore not "further"...so you could argue moz should not be required to deliver on reconnect to same session, but I still say moz has failed its QoS2 promise to me if it doesnt, because I was and still am a subscriber to that topic within the same session that it knows it has not sent me an instance of, no matter where it came from!

IF that is correct and there is no bug in moz then it means it is physically impossible to write an out-and-back QoS2 test program with both RX and TX on the same device...or more accurately in the same session. As my example (and my many wireshark logs) prove conclusively that message 3 is NEVER sent on reconnection. Thus my tester "misses" a transaction pair and claims a QoS2 failure. Ironically, if I had the TX and RX in two separate programs, the RX-only - not having crashed - would surely have received it and stayed "in-step" and claim continuous QoS2 success? I haven't tested this yet, and will wait to see if I get a heplful answer here first to make sure I am not missing something (other than RX[3]:) )

BUT that seems to contradict

The Session state in the Server consists of:

· The existence of a Session, even if the rest of the Session state is empty.

· The Client's subscriptions.

· QoS 1 and QoS 2 messages which have been sent to the Client, but have not been completely acknowledged.

· QoS 1 and QoS 2 messages pending transmission to the Client.

· QoS 2 messages which have been received from the Client, but have not been completely acknowledged.

As I see no way to argue that my missing RX3 is not "QoS 1 and QoS 2 messages pending transmission to the Client." whether "further" or otherwise!

Finally, on the same point, when moz CONNACKs with session set, what does that actually mean? I take it to mean that it thinks it has some stored session needing replay, in which case the next thing I see should be the incoming pubrels etc. What does it mean when session==true but nothing is sent by server? Is it to tell me it thinks I should have some stored session data needing replay? If so, why? I already know that because - like a good boy- I have stored session state which i replay on reconnection, so I dont need to be told, To me: session==true should mean only one thing. "stand by for some incoming replay requests"..,if it DOES then again. there's a bug as I see session==true and nothing incoming on most occasions, If it doesn't, then again I'm confused. which I readily concede is the most likely answer..: but:

Should mosquitto send TX[3] on "dirty session" reconnection in the above example? If not, why not?

Answer 1

Solution is to ad persistent true to mosquitto.conf file