[英]DELETE without lock in MariaDB/MySQL?`(InnoDB)
As far as I know, DELETE FROM ... WHERE
issues a lock on the table. 据我所知,
DELETE FROM ... WHERE
在表上发出锁定。
Now, I have a huge InnoDB table in MariaDB with a size of 1TB+ that is actively being used, and having it locked until the entire database has been searched for rows to delete is no option. 现在,我在MariaDB中有一个巨大的InnoDB表,其大小为1TB +,正在被使用,并且锁定它直到整个数据库被搜索到要删除的行是没有选择的。
Is there any way how I could delete rows matching certain criteria without locking it while the delete is happening? 有没有什么方法可以删除匹配某些条件的行而不会在删除时锁定它?
Here are some more specifics for this case: 以下是针对此案例的更多细节:
innodb_buffer_pool_size = 20G
innodb_buffer_pool_size = 20G
INSERT
s and SELECT
s at all times INSERT
和SELECT
data (BIGINT id, LONGTEXT data)
(where data
is a big chunk of JSON. I know that this is not a perfect relational database model, but the JSON comes from a third party, it's pretty complex, and could contain structure changes from the third party at any time and without notice) data (BIGINT id, LONGTEXT data)
的结构data (BIGINT id, LONGTEXT data)
(其中data
是JSON的一大块。我知道这不是一个完美的关系数据库模型,但JSON来自第三个派对,它非常复杂,可能随时包含来自第三方的结构变更,恕不另行通知) SELECT
s. SELECT
的某种“索引”。 (Simplified example, this could have a structure like data_index (BIGINT id, INT userId, INT itemId, BIGINT timestamp)
, so I could use SELECT
on userId and itemId, and join on the actual data. (timestamp is the unix timestamp in milliseconds) data_index (BIGINT id, INT userId, INT itemId, BIGINT timestamp)
这样的结构,所以我可以在userId和itemId上使用SELECT
,并加入实际数据。(timestamp是unix时间戳,以毫秒为单位) ) To fulfill the task, I would naturally come up with this simple query: 为了完成任务,我自然会想出这个简单的查询:
DELETE `data`, `data_index`
FROM `data_index`
LEFT JOIN `data` ON `data`.`id` = `data_index`.`id`
WHERE `timestamp` > (NOW() * 1000) - (7 * 24 * 60 * 60 * 1000)
But this would probably lock the tables for a pretty long time. 但这可能会锁定表格很长一段时间。 How could I accomplish the same task without locking the tables, so the database remains functional for other
SELECT
and INSERT
queries? 如何在不锁定表的情况下完成相同的任务,这样数据库仍可用于其他
SELECT
和INSERT
查询?
No, you can't DELETE without locking the rows examined. 不,如果不锁定已检查的行,则无法删除。
But you can minimize the number of rows examined by creating an index on the timestamp
column you are searching. 但是,您可以通过在要搜索的
timestamp
列上创建索引来最小化检查的行数。
This will also create gap locks against the potential rows you might try to insert at the end of the table, to ensure new rows don't affect the DELETE. 这还将针对您可能尝试在表的末尾插入的潜在行创建间隙锁定 ,以确保新行不会影响DELETE。
In InnoDB, ordinary write locks like those created by DELETE don't block reads. 在InnoDB中,像DELETE创建的普通写锁不会阻止读取。 Concurrent transactions can still read the rows—even the rows you're deleting.
并发事务仍然可以读取行 - 甚至是您要删除的行。
Ordinary write locks don't lock the whole table. 普通的写锁不会锁定整个表。 Well, it locks the table with an intention lock which just prevents other table locks, like those required by ALTER TABLE or DROP TABLE.
好吧,它使用意图锁来锁定表,这意味着阻止其他表锁,如ALTER TABLE或DROP TABLE所需的那些。 In other words, you can't ALTER/DROP a table while it has any reads or writes in progress.
换句话说,在进行任何读取或写入操作时,不能对表进行ALTER / DROP操作。
You might like my presentation: InnoDB Locking Explained with Stick Figures . 你可能会喜欢我的演示文稿: InnoDB Locking用Stick Figures解释 。
(NOW() * 1000) - (7 * 24 * 60 * 60 * 1000)
does not look like a valid time. (NOW() * 1000) - (7 * 24 * 60 * 60 * 1000)
看起来不像是有效时间。 It is 20170519568613000
, which looks like a mixture of DATETIME
and some kind of milliseconds. 它是
20170519568613000
,看起来像DATETIME
和某种毫秒的混合。 Perhaps you wanted UNIX_TIMESTAMP() * 1000 - (7 * 24 * 60 * 60 * 1000) = 1494742589000
. 也许你想要
UNIX_TIMESTAMP() * 1000 - (7 * 24 * 60 * 60 * 1000) = 1494742589000
。
How many rows are you expecting to delete? 您希望删除多少行? If it is a large number, then consider partitioning, or deleting in chunks
如果它是一个大数字,那么考虑分区或删除块
Maybe i am wrong, but on https://dev.mysql.com/doc/refman/5.7/en/innodb-locks-set.html i have read that it makes row lock, not table lock. 也许我错了,但是在https://dev.mysql.com/doc/refman/5.7/en/innodb-locks-set.html上我已经读过它会使行锁,而不是表锁。
Anyway you can try 无论如何你可以试试
DELETE ... FROM ... WHERE ... LIMIT x DELETE ... FROM ... WHERE ... LIMIT x
And execute as many times as needed. 并根据需要执行多次。 Between executions other queries can enter and minimize impact.
在执行之间,其他查询可以进入并最小化影响。 Of course, make this job on low load hours.
当然,在低负荷时间完成这项工作。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.