使用 DMS 迁移 MariaDB 数据以排除旧记录并减少 RDS 大小
[英]Migrating MariaDB data with DMS to exclude old records and reduce RDS size
问题描述
I have a MariaDB database with 1TB of data that I need to delete old records from.
我有一个包含 1TB 数据的 MariaDB 数据库，我需要从中删除旧记录。
The total deletion numbers will left the database with less than half of it's actual size so i thought that could be simpler if i migrate to a new RDS, using DMS and its filter functionality to copy only the records that i need and also replicate ongoing changes.总删除数量将使数据库的实际大小不到一半，所以我认为如果我迁移到新的 RDS，使用 DMS 及其过滤器功能仅复制我需要的记录并复制正在进行的更改，这可能会更简单.
The second reason to migrate is because i can't "give back" the free storage space after deletion to reduce costs, so amazon suggests to migrate to a smaller RDS.迁移的第二个原因是因为我无法将删除后的空闲存储空间“归还”以降低成本，因此亚马逊建议迁移到较小的RDS。
Considering this scenario, some doubts emerged.考虑到这种情况，出现了一些疑问。
1 - Its said that DMS does not create all the table objects. 1 - 据说 DMS 不会创建所有表对象。 Do i have to worry about the auto increment IDs?我是否需要担心自动递增 ID？ I mean, DMS will filter out records from the beggining of the sequence while migrating, so the DB will keep incrementing from the last ID when inserting new records?我的意思是，DMS 会在迁移时从序列的开头过滤掉记录，因此在插入新记录时，数据库会从最后一个 ID 开始递增吗？
2 - When exactly do I need to create the constraints, FKs and other non created objects? 2 - 我什么时候需要创建约束、FK 和其他非创建对象？
3 - Not all tables will have records being filtered. 3 - 并非所有表都会过滤记录。 Is it better to separate them in different migration tasks?在不同的迁移任务中将它们分开是否更好？
4 - I've never migrated anything using this tools. 4 - 我从来没有使用这个工具迁移过任何东西。 Is there any kind of hidden trap that i need to worry about?有什么隐藏的陷阱需要我担心吗？
OBS: The migration will be from AND to MariaDB OBS：迁移将从 AND 到 MariaDB
2 个解决方案
解决方案1
0 2020-01-05 21:11:21
1 - Before migrating I think it is advisable to delete the old records. 1 - 在迁移之前，我认为最好删除旧记录。 This will make it easier to migrate.这将使迁移更容易。 Maybe that will help you ...也许这会帮助你...
https://pt.stackoverflow.com/questions/139437/remover-registros-antigos-mysql-automaticamente
2 - Do not create restrictions before deleting old records. 2 - 在删除旧记录之前不要创建限制。 Because that can make mistakes.因为那可能会出错。
3 - It is good to separate the tables, both to delete the records and to import. 3 - 最好将表分开，以便删除记录和导入。 And that makes your migration faster in loading.这使您的迁移加载速度更快。 Since it has 1TB.因为它有1TB。
4 - Since you have never done this kind of migration, I recommend focusing on deleting the records so it is easier to import into your new system. 4 - 由于您从未进行过这种迁移，我建议您专注于删除记录，以便更轻松地导入新系统。
Good luck!祝你好运！ Send news发送新闻
解决方案2
0 2020-01-13 05:57:37
If you can dump, move, and load all but the biggest table by normal means, then do the following for the biggest for the dump part:如果您可以通过正常方式转储、移动和加载除最大表之外的所有表，则对转储部分的最大表执行以下操作：
mysqldump ... --where='...' ...
The where clause should be crafted to keep the rows you want to keep . where子句应当制作，让你想保留的行。
It sounds like you want to delete half of a big table.听起来你想删除一个大表的一半。 Such a DELETE can be quite problematic.这样的DELETE可能会很成问题。 By SELECTing the ones to keep, there should be a lot fewer hassles.通过SELECTing要保留的那些，应该会少很多麻烦。
Be sure to have innodb_file_per_table=ON in the new system.确保在新系统中有innodb_file_per_table=ON 。
声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.