Mosquitto with persitence as MQTT Forwarding Broker

Question

We want to use Mosquitto MQTT as Message Broker on a number of OpenWRT gateways to forward "local" anonymous MQTT publications to a central RabbitMQ/MQTT cluster with authorization.

Our Problem:

Mosquitto does not reliable forward queued messages when restarted (persistance does not work)

Running mosquitto version: 1.4.15

current config

This is the last tested config:

port 1883
persistence true
persistence_file /mosquitto.db
persistence_location /etc/mosquitto

autosave_interval 1
autosave_on_changes true

allow_anonymous true

connection iotcluster
address ip:1883

notifications false
keepalive_interval 300
restart_timeout 30
start_type automatic

clientid rabbitmqtt
username user
password password

topic mqtt out 2
try_private true

As long as all systems running and online - everything works as expected - any message published locally via:

mosquitto_pub -h localhost -p 1883 -t mqtt -m "Test-Bridge-Online"

will be forwarded and published to the RabbitMQTT.

connection lost

When we produce a connection lost (eg disconnect cable) and re-establish that connection, the messages received in the meantime won't be automatically forward from Mosquitto to RabbitMQTT

BUT - when sending a new publish message in addtion - Mosquitto will send out also the queued messages???

Mosquitto restart

If we restart the Mosquitto during the connection lost, the queued messages are all lost - NO PERSISTINACE options worked so long.

Please help

We tried different options of autosave_interval , QoS 0/1/2 ond other option combination - but in anyway - on restart Mosquitto - all messages are lost - NO PERSISTANCE in any way

Answer 1

After running an update to mosquitto version 1.6.10 still no success.

I could isolate the main problem - the database did not save when seetings:

persistence true
persistence_file /mosquitto.db
persistence_location /etc/mosquitto

autosave_interval 1
autosave_on_changes true

Not sure why but could not manage to invoke a save of database by the above settings.

The database save on SIG-EXIT but not when pushing new message.

So I changed the settings to:

persistence true
persistence_file /mosquitto.db
persistence_location /etc/mosquitto

autosave_interval 300
autosave_on_changes false

and invoke a request to save the database after pushing the message:

# publish a new message
mosquitto_pub -h localhost -p 1883 -t mqtt -m "Test Offline 100" -q 1
# send signal to save database
killall -SIGUSR1 mosquitto

When using this - all kind of messages where queued in an state of the connection and get delivered.

That's the final configuration:

user mosquitto
port 1883

allow_anonymous true
queue_qos0_messages true

persistence true
persistence_file mosquitto.db
persistence_location /etc/mosquitto/
autosave_interval 300
autosave_on_changes false

connection ConName
address remote-ip:1883
bridge_protocol_version mqttv31
clientid ClientName
username user
password password
cleansession false
try_private false
retain_available false

start_type automatic
restart_timeout 60
keepalive_interval 120

notifications true
notifications_local_only true
notification_topic mqtt

topic mqtt out 1