MySQL - query from slave and insert on master

Question

I used to run this command to insert some rows in a counter table:

insert into `monthly_aggregated_table`
select year(r.created_at), month(r.created_at), count(r.id) from 
raw_items r
group by 1,2;

This query is very heavy and takes some time to run (millions of rows), and the raw_items table is MyISAM, so it was causing table locking and writes to it had to wait for the insert to finish.

Now I created a slave server to do the SELECT .

What I would like to do is to execute the SELECT in the slave, but get the results and insert into the master database. Is it possible? How? What is the most efficient way to do this? (The insert used to have 1.3 million rows)

I am running MariaDB 10.0.17

Answer 1

You will have to split the action in 2 parts with a programming language like java or php in between.

First the select, then load the resultset into your application, and then insert the data.

Another optimization which you could do to speed the select up is add one new column in your table "ym_created_at" containing a concatenation of year(created_at) and month(created_at). Place an index on that column and then run the updated statement:

insert into `monthly_aggregated_table`
select ym_created_at, count(r.id) from 
raw_items r
group by 1;

Easier and might be a lot quicker since not functions are acting on the columns you are using the group by on.