Sending emails in PHP and using Google Apps mail as SMTP is too slow

Question

Currently I've been using PHP and the Zend Framework to send emails with SMTP and GMail's SMTP servers. That's because I have a Google Apps domain setup to handle all my emails so it looks like my emails are coming from mailer@mycompany.com.

Now when a user registers or does something which requires my web server to send an email to them this can take 5 seconds or so on the PHP side to initiate the SMTP connection and send the email to them. The process would be something like this:

1. User completes registration form
2. User submits form
3. PHP script saves information to database and sends them an email (5-10 seconds)
4. User redirected to another page

Now when they've just submitted the form I can't have a massive delay from my web server while the user sits there for 10 seconds looking like it's loading. Basically the send email process is single threaded so the server can't do anything else until it's completed sending the email which is anywhere between 5-10 seconds, sometimes 20 seconds if there's an error with the email address so this is far too long.

How to people normally get around issues like this?

Originally I tried a few things:

1. Using another library like SwiftMailer. Still same problem. I think it's the SMTP/TLS handshake which takes extra time.

2. Storing the email data in a database and then using fsockopen to initiate an asynchronous request to another page which would then get the email data from the database and send the email. Meanwhile my PHP script could continue synchronously and the user would get redirected to the next page immediately after the form submission. Problem was there was no way to confirm whether the email was sent, or if it failed. I could set a flag on the database table I suppose but I wanted to provide instant feedback to the user that their email had been sent. That meant on the next page, firing off an Ajax request after 10 seconds to another page which would check the flag on the database table to see if it was sent or not, then show the response on the page to the user. This wouldn't work if they entered the email wrong and it couldn't send the email, as the response from GMail would sometimes take 20 seconds for an error case, then the database flag wouldn't be set.

3. Another option I came up with was to put the email data in the $_SESSION array, so after submitting the page and the user is on the redirected page it would send an ajax request off to another page which would get the session data and send the email, then send a response back to the page the user was on and tell them the email was sent. This works pretty well, but I feel it's probably not the best way.

So I'm wondering what's the best practice way to do it? Is GMail too slow for this kind of thing as it has to do the TLS handshake? Should I just queue up all the emails into the database and run a CRON job every few minutes to send them off, then remove them from the database once sent? Problem with that is there's no instant feedback given to the user that their email was sent correctly. Or do I not need to bother with that anyway? They either get the email or not.

Answer 1

Have you considered using your host's email smtp server and setting the from address manually? This has typically worked for me. If that is not an option, my first reaction would be option 3 - this is similar to what Google mail does when you select the option for delayed sending in their web ui.

Answer 2

What about writing an apps script and using it as a webapp just to handle the e-mail. You can use HTTP Get with params to have apps script send the e-mail using the base mail service. MailApp.sendEmail()

I'm a self taught apps script guy so I'm just speculating here, but in theory it should work. I think it would look something like this.

You should be able to pass the keys at the end of e.parameter through the webApp url.

function doGet(e){
var userEmail = e.parameter.email;

var body = 'message here';//could even pass params for the body also.

MailApp.sendEmail(userEmail, 'Subject', 'body', {noReply: true});
}

All our stuff is built on apps script and I have dozens of email notifications going out, and in apps script its instant.

Hope this helps.

Answer 3

I had this exact problem, What I actually do is that I just queue them in database and I have a cronjob that runs every minute to send emails queued in database.

by the way, many mid-sized sites do this, and tell you you should receive an email within (5 minutes) or shortly ........etc

The benefits of saving in db is that you can add an extra column to indicate if it's sent or not and to retry if it's not sent in the next cronjob.

---

Update for locking the Cronjob:

Please notice that this is a sequential logic only and notice that step 1,2, and 4 are about Cron #1

1-Cron #1 runs at 12:05, reads the locking table and finds it empty so it creates a db record in a pre-created table "locking" with 1 field "cronlock": holding the value 12345678910, which is the current time stamp.

2-Cron #1 at 12:06 (Still sending emails, but it completed 6 emails so far) so it updates the record of locking again with the current time-stamp to extend the locking period.

3-Cron #2 runs at 12:07, reads the locking table and find a value, compares this value with the current timestamp if the difference is less than 120 seconds, then it terminates itself.

4-Cron #1 finishes sending some emails at 12:08, it deletes the locking record and quits.**

5-Cron #3 runs at 12:09 and finds no locking record so it creates a new one and starts sending.... and so on.

Answer 4

Actually I'm using same components to send emails. But I'm sending emails in background using Zend_Queue in this case visitors don't have to wait. This is how I accomplish this task ( Zend_Queue is initialized in Bootstap.php ) :

1.Creating job

class My_Job_SendEmail extends My_Job
{
    /**
     * Message subject:
     * 
     * @var string|null 
     */
    protected $_subject = null;

    /**
     * Message body:
     * @var string|null 
     */
    protected $_message = null;

    /**
     * Message To:
     * @var string|null 
     */
    protected $_to = null;

    public function __construct(Zend_Queue $queue, array $options = null)
    {
        if (is_array($options)) {
            $this->setOptions($options);
        }

        parent::__construct($queue);
    }

    public function job()
    {
        $mail = new Zend_Mail();
        $mail->addTo($this->_to);
        $mail->setSubject($this->_subject);
        $mail->setBody($this->_message);
        return $mail->send();
    }
}

2.Source of My_Job

3.Adding job to the queue (probably somewhere inside your service)

$backgroundJob = new My_Job_SendEmail(Zend_Registry::get('queue'), array(
    'to'      => $to,
    'bcc'     => $bcc,
    'subject' => $subject,
    'message' => $message,
));

$backgroundJob->execute();

4.See this answer, for an example of background script.

5.Add background script to the cronjob

*/1 * * * * php /path/to/project/background/emailQueue.php