Implementation of sending emails through a cron job

Question

Consider the following 'pending_send_confirm' table in a database, used to store the email addresses that I need to send email to (the final user will confirm their subscription through that email):

id| address |occupied|
0 |em1@s.com|0       |
1 |em2@s.com|0       |
2 |em3@s.com|0       |

When a user subscribes (to my newsletter, for example), I want to show a message saying that everything went fine almost immediately. I don't want him to wait for the server to send the confirmation email to his address before showing a message.

That is the reason that I setup a cron job running every minute in order to manage emails that have to be sent. According to the table I came up with (shown above), this is some pseudocode sending emails to each address, 1 by 1:

//child_script.php
//fetch the entries from the database
while ($entry = $result->fetch_assoc()) {
    if ($entry['occupied']) {
        /*
         * Another instance of this script has 'occupied' this address, which
         * means that it is currently trying to send a confirmation email to
         * this address. So you know that another instance is working with this
         * email address, so skip it.
        */
        continue;
    }

    /*
     * Entry is not occupied, occupy it now in order to prevent future instances
     * of this script to attempt to send additional confirmation email to this address
     * while this instance of the script tries to send the confirmation email to this address.
     * occupied=1 means that an attempt to send the confirmation email is under the way
    */

    occupyEntry($entry['id']); //sets 'occupied' to 1

    if (sendConfirmationEmail($entry['address'])) {

        /*
         * Email was sent successfully, move the email address from the 'pending_send_confirm' to the 
         * 'pending_confirmation_from_user' table.
        */

        moveToConfirmPendingFromUserTable($entry['id']);
    } else {

        /*
         * Failed to send the email, unoccupy the entry so another instance of the script
         * can try again sometime in the future
        */

        unoccupyEntry($entry['id']); //sets 'occupied' to 0
    }


}

Code without comments for readability:

//child_script.php
while ($entry = $result->fetch_assoc()) {

    if ($entry['occupied']) {
        continue;
    }

    occupyEntry($entry['id']);

    if (sendConfirmationEmail($entry['address'])) {
        moveToConfirmPendingTable($entry['id']);
    } else {
        unoccupyEntry($entry['id']);
    }
}

Is this a solid solution in order to prevent duplicate emails to be sent out? I worry that 2 instances of the script may find at the "same time" that $entry['occupied'] is 0 for a specific id and try both to send an email to that address.

Another solution would be to use flock ( How can I ensure I have only one instance of a PHP script running via Apache? ) so as to ensure that only 1 instance of my script is running.

I can see many problems with the flock implementation, though. For example, what happens if my script crashes prior to calling fclose($fp) ? Will the next instance of my script be able to continue or will it see it as another instance of the script running (aka what will the flock function return to the new instance of the script)? Another problem is that my script sends emails one-by-one. This means that if I have 100 emails to send and I send them in 3.5 minutes, then the next instance will start 4 minutes AFTER the 1st instance STARTS. This means that if a subscriber chooses to subscribe the moment that the 1st instance starts then they will have to wait 4+ minutes in order to receive their confirmation email. If I allow scripts to work in parallel, then the emails will be sent out way faster. So I would prefer to be able to have multiple instances of the same script, sending emails at the same time.

On a side note, if my script, using the 'occupy' method works just fine, Will I be able to have another script manage the number of simultaneous instances that will be launched? For example, will I be able to do the following? :

//master_script.php

/*
 * Launch many instances of child_script.php.
 * If there are 901 to 1000 emails, then start 10 instances etc
*/

launch_n_instances( ceil(number_of_non_occupied_entries()/100.0) );

What is the right way to deal with this problem?

Answer 1

In another table create a field called "cron_running" or something similar and set it to true at the top of the cron script. At the end of the script set the field back to false. This will allow you to check if the cron script is still running if it happens to overlap and not continue.

Answer 2

There are many questions coming up, when reading about your problem:

You say:

I don't want him to wait for the server to send the confirmation email to his address before showing a message.

First, the latency being added by sending out a conformation mail should be very short.

Second, a newsletter subscription seems much like an "email thing" to me. How can you tell, the "subscription went fine", if you didn't get some confirmation?

The other problem you mention in fact boils down to how to do bookkeeping of some status - in this case "email sent" - for multiple processes in a database. Given your database has an appropriate isolation level , ie you can read only things that aren't being changed the database takes care of this.

Your "occupy" field could instead be called email_status with values,

0 - new
1 - processing
2 - created
3 - confirmed 

The last point is, that this bookkeeping is just a transient status. You don't want to keep all those redundant "confirmed" entries in your database, just to have the information about possible errors on the way from "new" to "confirmed".

So you might even use a distinct table, consisting of id and status where the last step of processing is deleting them. That way your additional table would just contain the ids not fully processed.