简体   繁体   中英

MySQL purging shell script

I have a mysql table with 120 million rows and I want to write a shell script to start purging some of that useless information in that table that's not needed. Problem is I'm super new to shell scripting.

I have a datetime column with a unix timestamp in it. I want to delete every row that's not within the last 2 months since I've recently enacted a data retention policy that will allow me to only keep 2 months of certain data.

TL;DR Need to build a shell script that deletes all rows up until the last 2 months of data by using the unix timestamp in the datetime column.

UPDATE: Here's my new shell script

#!/bin/sh

i=1
while [ "$i" -ne 0 ]
do
  i=mysql -h 127.0.0.1 -u halo_ylh -pCa5d8a88 halo_twittstats < mysqlpurge.sql
  sleep 10
done

Wouldn't it be easier to just use the current_timestamp and unix_timestamp function to execute:

DELETE FROM Table1 
 WHERE datetime < unix_timestamp(current_timestamp - interval 2 month)

To run it from the shell you could put that command in a file script1.sql and run it using the mysql command line tool:

mysql db_name < script1.sql
mysql --host ? -user ? -password ? {DB_NAME} < dump_script.sql > data.dat 

dump_script.sql will have your SELECT statememt to retrieve the data you want archived. it will store the output in data.dat

then

mysql --host ? -user ? -password ? {DB_NAME} < delete_script.sql

delete_script.sql will cotain the DELETE statement with the same WHERE clause as in dump_script.sql.

Make sure you lock the table so that nothing writes in between the two script exections that could make it into the WHERE clause to avoid phantom inserts that would be deleted by the delete script yet not be included in the dump script.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM