Multiple spools with Swiftmailer and Symfony2

Question

What is the best approach for sending the highest email rate possible with Swiftmailer?

We own an email automation tool and sometimes there are single sendouts of 40.000 emails. Our average rate with the spool:send command is of ~50 emails/min. I've tried copying the same command on the cron 5 times and it worked (ie it was sending ~250 emails/min), but it looks like the SMTP server got dizzy, because some contacts were receiving emails with another contact's information (any idea on what could be causing that?).

So now I was thinking about setting up 5 different mailers that spools the emails on different folders and running the 5 commands with a cron, each one for one of those mailers. Should it work? Any other recommended solution?

Answer 1

If you're sending 250 emails per minute, then you need something more resilient than the cron and the Swiftmailer spool. It will be hard to scale, a nightmare to debug, and not very inflexible. The Swiftmailer spool is great if you're sending no more than a couple of emails a minute, but any bigger than that and it's hard to scale and a nightmare to debug.

Instead, use a job queue like PHP Resque or Rabbit MQ (both are open source). You can replicate the 'spool' by having a queue of emails that need to be sent, and you can add multiple workers and queues. You could also have a second queue that actually adds the jobs to the first queue.

The advantage is that Rabbit MQ comes with a manager interface, so you can see things like how many emails are being sent, how many are failing, etc. Also, it's easier to scale up and down by adding and removing workers when you're under heavy load, for example.

Kacper from Sensio Labs actually gave a talk on Rabbit MQ with Symfony last year - http://www.slideshare.net/cakper/2014-0821-symfony-uk-meetup-scaling-symfony2-apps-with-rabbit-mq .