Pushing binary log files to MySQL slave

Question

I am using MySQL to store log data from a number of sensors on-board a boat, and would like to replicate that database to a read only instance of MySQL on a web server.

The web server has a static ip, but the log server will have a dynamic ip address from behind different routers depending where the boat is.

As far as I can understand from reading around, binary log files are usually requested by a slave from the master, which is not feasible in my application.

Is it possible to 'push' binary log files to a remote slave to achieve updates for the read only MySQL database?

Any pointers other than using MySQL dump files with scripts would be much appreciated.

Answer 1

Unfortunately MySQL replication depends on IP address of the master. So if master changes IP address you will get replication error on replica. But this is not so bad because MySQL replication is quite robust.

But you must set long enough "expire_logs_days" on your master to make sure you will have all bin logs available even after some longer time without connection - especially if your boat will be long time without internet connection. Of course to store huge amount of bin logs you need to have huge disks - or better disk array to avoid errors.

And master must be able run without crashes so there will be no problems with the continuity of bin logs on master.

Maybe you will find some inspiration in following possibilities:

Scenario 1 - try to use VPN

I am not network guru but you could try to use VPN from your master on boat to replica using OpenVPN or ZeroTier or something similar using other internet server with static address which will be accessible for both machines. This way you will create your virtual network and will address master with IP in this VPN. May probably require to repeatedly modify VPN from your master when it moves and changes its dynamic IP.

Scenario 2 - change IP of master on replica when necessary:

I am not network guru but let's presume new dynamic IP of your master will last long enough and master will be accessible under this IP from outside using standard internet communication. And either replica will be able to find new IP or you can send new IP of master to the replica to some listener or web service. If so then every time IP changes and you get replication error on replica you can simply use following commands on replica:

STOP SLAVE; CHANGE MASTER TO MASTER_HOST='xxx.xxx.xxx.xxx'; START SLAVE;

This works because replica knows which last position in bin log was successfully applied. So when you change only master IP and start replication again, replica will simply continue.

Of course if master will not be accessible under some dynamic IP from outside then replica will not able to contact it...

Scenario 3 - batch shipment of datafiles

If you will not be able to open VPN or access MySQL master on the boat from outside or ensure internet connection lasting long enough or use disks big enough for all archived bin logs you can always do hot backups on running master using Percona XtraBackup.

This tool makes consistent backup of all datafiles with applying of all changes made during the backup. You can tar and gzip it and send to some cloud storage. This way you will not lose any data and will have really consistent backup.

Advantage of this solution is - on target machine you simply stop mysql, delete existing old data files, untar/unzip new datafiles and start mysql and here we go. No need to make usually very time consuming restore of the dump file.