mysql-python-replicator binlog start pos for specific schema

Question

I'm using mysql-python-replicator to dump data from mysql for specific database ,so I can create a BinglogStream with parameters only-schema:

stream = BinLogStreamReader(connection_settings=mysql_settings,
                            server_id=1,only_schemas='tpch',
                            log_file='mysql-bin.000001',log_pos=1
                            )

I have the parameters log_file='mysql-bin.000001' ,and log_pos=1 ,because I have no idea where the event position start,so any ideas to find the start event position for a database in mysql-binlog ?

Answer 1

Guessing the first binlog position won't work. This file and position might not exist on the server because Mysql automatically rotates those logs based on its configuration.

The library internally knows how to start from the beginning. Simply leave out the 3 parameters needed for resuming (or set them False / None):

if binlog_filename is None or binlog_position is None:
    # Connecting to MySQL binlog (from beginning)...
    binlog_stream = BinLogStreamReader(
        connection_settings=mysql_settings,
        server_id=1,
        blocking=True
    )
else:
    # Continuing from binlog_filename and binlog_position
    binlog_stream = BinLogStreamReader(
        connection_settings=mysql_settings,
        server_id=1,
        resume_stream=True,
        blocking=True,
        log_file=binlog_filename,
        log_pos=binlog_position
    )

Answer 2

At first you could start it with resume_stream=False,log_file=None,log_pos=None,blocking=False which will presumably start reading the binLogEvents from the beginning. On a subsequent iteration you should somehow (in a db or in memory) record the latest log_file and log_pos, which you can read from the BinLogStreamReader object. Since blocking parameter is set to False the binlog file will close once out of events or when closed manualy by you, and you can recreate it but now with different parameters which you read from you place of recording.