[英]Very slow migrations
I have a (KVM) Debian 8 VM server it's running MySQL 5.5.58 and PHP7.1 locally I have a similar machine (Vagrant VM, Debian 8, VirtualBox). 我有一台(KVM)Debian 8 VM服务器,它在本地运行MySQL 5.5.58和PHP7.1。我有一台类似的机器(Vagrant VM,Debian 8,VirtualBox)。 Locally my migrations run < 20 seconds if I dump them into a SQL file and import the SQL file it takes about 2-3 seconds.
如果将它们转储到SQL文件中并导入该SQL文件,则在本地运行的迁移时间少于20秒,这大约需要2-3秒。
However on the (KVM) Debian 8 VM server migrations run between 10-20minutes. 但是,在(KVM)Debian 8 VM服务器上的迁移运行时间为10-20分钟。 I have tried increasing CPU count and RAM to no avail.
我尝试增加CPU计数和RAM均无济于事。
There are however two things that greately improve performance but still not really usable: 但是,有两件事可以极大地提高性能,但仍然无法真正使用:
innodb_flush_log_at_trx_commit=0
and run migrations this reduces migration run time down to about 4-5 minutes which is still not usable. innodb_flush_log_at_trx_commit=0
并运行迁移,这会将迁移运行时间减少到大约4-5分钟,这仍然无法使用。 my.cnf my.cnf
[mysqld]
sync_binlog = 0
federated = 1
innodb_use_sys_malloc = 0
innodb_file_per_table = 1
innodb_stats_on_metadata = 0
innodb_buffer_pool_instances = 1
query_cache_type = 0
innodb_buffer_pool_size = 1G
innodb_log_file_size = 768M
innodb_flush_log_at_trx_commit = 2
innodb_flush_method = O_DIRECT
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
skip-external-locking
bind-address = 127.0.0.1
key_buffer = 16M
max_allowed_packet = 16M
thread_stack = 192K
thread_cache_size = 8
myisam-recover = BACKUP
#max_connections = 100
#table_cache = 64
#thread_concurrency = 10
query_cache_limit = 1M
query_cache_size = 16M
Filesystem: 文件系统:
Filesystem Size Used Avail Use% Mounted on
/dev/vda 32G 6.0G 25G 20% /
udev 11M 0 11M 0% /dev
tmpfs 422M 17M 406M 4% /run
tmpfs 1.1G 0 1.1G 0% /dev/shm
tmpfs 5.3M 0 5.3M 0% /run/lock
tmpfs 1.1G 0 1.1G 0% /sys/fs/cgroup
tmpfs 211M 0 211M 0% /run/user/2031
What I have tested and checked: 我测试和检查的内容:
getUp
method from Migrator
. Migrator
定时执行Laravel getUp
方法。 Specifically timed reading migration file from disk, creating a new class and executing the query all seems to be executing < 1s per migration used microtime(true)
. used microtime(true)
迁移都执行<1s。 CREATE TABLE
statements. CREATE TABLE
语句。 Note: This happens in all ways of using MySQL I tested with MySQL installed on host. 注意:在使用我在主机上安装了MySQL并测试过的MySQL的所有方式中,都会发生这种情况。 I installed docker mysql same issue.
我安装了docker mysql同样的问题。 I installed vagrant instance and mysql inside that same issue.
我在同一问题中安装了vagrant实例和mysql。
This innodb_flush_log_at_trx_comit
seems to make the most difference indicating I messed up something with mysql just not sure why. 这个
innodb_flush_log_at_trx_comit
似乎起了最大的作用,表明我弄乱了mysql,只是不确定为什么。 Migrations each runs as it's own transaction seems something is causing wait time here not sure what else to check. 迁移每次运行都是因为它自己的事务 ,这似乎是导致等待时间的原因,在这里不确定要检查什么。
Your raw SQL most likely has a mess of INSERT
queries in it. 您的原始SQL很可能在其中包含一堆
INSERT
查询。
Edit The SQL generated by MySQLDump certainly does have lots of inserts in it. 编辑 MySQLDump生成的SQL当然确实包含很多插入内容。 Populating a test dbms is often done from a version-controlled SQL file.
填充测试dbms通常是从版本控制的SQL文件完成的。 Here's some q&a about doing it in docker .
这是有关在docker中进行操作的一些问答 。
Wrap each series of 100 queries or so (the actual number doesn't matter very much) in transactions by putting START TRANSACTION;
通过放入
START TRANSACTION;
在事务中包装每系列100个左右的查询(实际数量无关紧要) START TRANSACTION;
at the beginning and COMMIT;
在开始和
COMMIT;
at the end. 在末尾。 That will get rid of the churning that goes with autocommit.
这将摆脱自动提交带来的麻烦。 You'll have a SQL file looking something like this:
您将拥有一个类似于以下内容的SQL文件:
START TRANSACTION;
INSERT ...;
... (more inserts)
INSERT ...;
COMMIT;
START TRANSACTION;
INSERT ...;
... (more inserts)
INSERT ...;
COMMIT;
START TRANSACTION;
INSERT ...;
... (more inserts)
INSERT ...;
COMMIT;
Avoid horsing around with innodb parameters; 避免使用innodb参数; the defaults are generally pretty good, and only need to be changed for large dynamic loads.
默认值通常很好,并且只需要为较大的动态负载进行更改即可。
And don't expect your VMs to perform as well as your laptop. 而且不要指望您的VM表现得比笔记本电脑好。 The days when servers were faster than laptops are gone (for most of us).
服务器比笔记本电脑快的时代已经过去了(对我们大多数人而言)。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.