What is the most efficient way to email a lot of users in PHP?

Question

My website sends an email out to my user's whenever someone PM's (Private Message) them. It says something along the lines of "Someone just sent you a message on SITE NAME, login to view it".

At the moment my site isn't very popular so my current way of doing this hasn't been a problem performance wise:

    // code that sends pm here...

    // code that stores pm in messages table here...

    // notify user by emailing them right away
    mail($user_email, $subject, $message);

Basically every time a PM is sent, the mail function is executed and the message is sent right there on the spot. So for every 100 messages sent the mail() gets called 100 times.

I am anticipating my site getting more popular and with more users come more PM's so I think my current way of doing it would become a performance nightmare. So I was thinking of doing it this way instead :

// get the last message that was accounted for by the last cron
$query = mysql_query(" SELECT `last_checked_id` FROM `settings` ");
$last_checked_id = mysql_fetch_array($query);


$user_emails = array();

// get all messages that were added to this table since the last time this cron ran
$query = mysql_query(" SELECT `recipient_id`, `message_id` FROM `messages` WHERE `message_id` > '$last_checked_id' ");

$i = 0;
while($row = mysql_fetch_array($query)) {

    if($i == 0) {
        // set this as the last message processed
        mysql_query(" UPDATE `settings` WHERE `last_checked_id` = '". $row['message_id'] ."' ");
    }

    // only put this user in the to email list if he hasn't been added already (since there can be multiple messages for the same user but we need to notify him only once) 
    if(!in_array($row['recipient_id'], $user_emails)) {
        array_push($user_emails, $row['recipient_id']);     
    }
}
$i++;

// send email to all users at one
mail(implode(',', $user_emails), $subject, $message);

I can set this script up as a cron and have it run every hour. All the emails are sent in one go with the mail function being called only once. The only drawback is user's aren't notified right away when they recieve a PM, but once within the hour. But that's not a big deal and something I can live with.

My questions are:

• is the cron method significantly better performance wise or is the increase negligable
• is this the way most big sites do it? or is there a better way, some established library maybye?

Thanks.

Answer 1

You can insert mails a mysql table (emailoutgoing)

And with a cron job running every "3" minutes, you can get unsent (100?) emails from table, send and mark row as sent.

Answer 2

If I understood correctly you would be sending exactly same email to every person? In that case you should use bcc field for email adresses or everybody would see other recipients but in that case you can have problems with anti-spam engines that might disregard your emails since they will not see their user as a recipient of email (not located in to or cc field ).

But you should really think about sending personalized email for each user (they will be much more eager if you add sneek preview into who sent a message and/or part of a message)

Answer 3

Firstly, read the link in Mike B's comment.

Secondly, your code has some entertaining failure modes - it could fail, mark all mails as sent, and nobody would ever know. A key thing about CRON and other unattended processes (including websites in general) is that you need to know when things go wrong. I'd consider putting the "update" statement right at the end, after you've sent the mail. Put in some kind of logging so you can keep track of what's going on.

Thirdly, please don't send this email to all these people in the "to" field - as Miro says, it's spam bait, but it also exposes users to more information than they should probably have - they know the email addresses of random other users of your site who happen to have had a PM within an hour of them. If you're building a dating site or social network, that's almost certainly going to make people angry.

Finally, other sites I've worked on that had similar requirements tended to have an "events" table, which was monitored by daemon/service style processes. The daemon would make smart decisions about what to do. We designed the solution to be suitable for running on multiple threads and even multiple machines, though in practice never needed this.

So, you might have an "events" table as follows:

eventID    eventDate    eventType     eventStatus     eventMeta
-------------------------------------------------------------------
1          1 Feb 2012        PM           NEW         <from>bob@banana.com</from><to>alice@passionfruit.com</to>
2          1 Mar 2012        PM           COMPLETE    <from>fred@pear.com</from><to>jennifer@avocado.com</to> 

The daemon constantly scans for records where eventStatus is NEW, sets them to "in progress", deals with them, and then sets the status to complete.

This allows you to see what's going on in the system - if a record is "in progress" for more than a second, something is broken. If the number of "NEW" records keeps growing, and the number of "complete" records doesn't, something is broken.