如何在多个MySQL语句中使用限制选择唯一记录

Question

I have 10 seperate php chron jobs running that select 100 records at a time from the same table using

SELECT `username` FROM `data` where `id` <> = '' limit 0,100

How do I ensure that each of these recordsets are unique? Is there a way of ensuring that each chron job does not select the same 100 records?

username is a unique if that helps.

Thanks

Jonathan

Answer 1

• You can either choose different 100 records:

  limit 100,100 , limit 200,100 ...

• Or choose 100 randomly:

  ...FROM data where id <> = '' ORDER BY RAND() LIMIT 0,100

• If you want to ensure that a record would not be chosen twice, you'll have to mark that record ("make it dirty"), so other cron jobs would be able to query only ones that were not chosen already. just add another boolean key called chosen , and mark it true after a given record was chosen. You'll have to run the cron jobs one by one, or use locking or mutex mechanism to ensure they won't run in parallel and race each other.

Answer 2

What you could do is 'mark' the records each job is going to use - the trick would be ensuring there's no race condition in marking them. Here's one way to do that.

create table job
(
    job_id int not null auto_increment,
    #add any other fields for a job you might want
    primary key(job_id)
);

# add a job_id column to data
alter table data add column job_id not null default '0', add index(job_id);

Now, when you want to get 100 data rows to work on, get a unique job_id by inserting a row into job and obtaining the automatically generated id. Here's how you might do this in the mysql command line client, easy to see how it is adapted to code though:

insert into job (job_id) values(0);
set @myjob=last_insert_id();

Then, mark a hundred rows which are currently 0

update data set job_id=@myjob where job_id=0 limit 100;

Now, you can take your time and process all rows where job_id=@myjob, safe in the knowledge no other process will touch them.

No doubt you'll need to tailor this to suit your problem, but this illustrates how you can use simple features of MySQL to avoid a race condition among parallel processes competing for access to the same records.