Sending a job to gearman from a mysql trigger on large updates

Question

I am using gearman's udf to send a background job to gearman from an after update trigger.

the documentation states the following :

The gman_do* functions take an optional third parameter, which is the unique job ID. This allows you to submit multiple jobs under the same unique ID, and they will be "merged" in the queue to be run only once. Note that gearmand will only merge jobs that are in queue or running, it doesn't keep track of unique IDs after a job finishes. For example, the following makes sure you only run the job once for each host:

SELECT gman_do_background("reverse", Host, Host) AS test FROM mysql.user;

Which makes it looks like it's ready to handle a large number of lines updates without flooding my gearman's queue.

But when I update 30k lines at once, my gearman daemon becomes unreachable for a while, and "netstat" shows ~30k connections on the 4730 port , which I guess explains why.

Any idea how to prevent the freeze to happen without slowing too much the update ? I only need the gearman job to be triggered once after the whole update query, but as far as i know, mysql/mariadb do not handle after statement triggers.

Answer 1

MySQL/MariaDB only supports triggers that execute FOR EACH ROW (you may have noticed this mandatory part of the syntax of a trigger).

I would never recommend calling a gearman UDF in a trigger anyway. You can't know in the trigger whether the update that spawned the trigger will be committed (or when it will be committed). The gearman service may receive notification about an update that it can't view, because it's not committed yet.

Instead, forget about the trigger. Write code in your application to do the update, and then confirm it succeeded and that the transaction is committed. Only then notify gearman — by calling the gearman API from your app, not by using a MySQL UDF.

Then you have the flexibility to make one notification for a batch of rows updated.

There's pretty much no good way to use any UDF in MySQL that causes side-effects outside of transaction scope, and there's no need to do so anyway.