The application goes offline when trying to send more than thousands of emails in Rails with AWS SES

Question

I have implemented a platform using rails, and the goal is to send thousands of emails to customers with one click. The concept is that an email array runs each loop and inside each loop runs send email functionality like below.

@emails = ['abc@gmai.com', 'abc@example.com'] # More than 3 thousands

@emails.each do |email|
    aws_email_sender(email, @email_subject, @email_body_html)
end

And the email function is like below:

def aws_email_sender(recipient, subject, htmlbody)
    sender = "hello@example.com"

    awsregion = "ap-west-1"


    # The HTML body of the email.
    htmlbodycontent = "#{htmlbody}"

    # The email body for recipients with non-HTML email clients.  
    textbody = "This email was sent with Amazon SES using the AWS SDK for Ruby."

    # Specify the text encoding scheme.
    encoding = "UTF-8"

    # Create a new SES resource and specify a region
    ses = Aws::SES::Client.new(region: awsregion)

    # Try to send the email.
    begin
        # Provide the contents of the email.
        resp = ses.send_email({
            destination: {
                to_addresses:  [recipient]
            },
            message: {
                body: {
                html: {
                    charset: encoding,
                    data: htmlbodycontent
                },
                text: {
                    charset: encoding,
                    data: textbody,
                },
                },
                subject: {
                charset: encoding,
                data: subject,
                },
            },
            source: sender,
        });

    
    # If something goes wrong, display an error message.
    rescue Aws::SES::Errors::ServiceError => error
        puts "Email not sent. Error message: #{error}"
    end
end

The email is sending well by AWS but my rails application has gone down like

A timeout occurred, error code 524

I couldn't get the breaking point, why has my application gone down every time?

Thanks in Advance

Answer 1

If 524 is an HTTP status code then it means...

Cloudflare was able to make a TCP connection to the website behind them, but it did not reply with an HTTP response before the connection timed out.

Meaning your Rails app is behind a Cloudflare proxy. Cloudflare received an HTTP request, forwarded it to your app, waited around for your app to respond, but your app never did. A more detailed explanation can be found here .

Probably because it's trying to send emails to 3000 people one-by-one.

There's two strategies to fix this.

Use Bulk Email

Since the content of the email is the same for everyone, use an email template to send bulk email using the #send_bulk_templated_email method.

You can send to up to 50 addresses at a time, so use #each_slice to loop through emails in slices of 50.

This will be more efficient, but your app will still be waiting around for 3000/50 = 60 AWS API calls. At worst it will still time out. At best the user will be waiting around for a form submission.

Use A Background Job

Anytime your app needs to do something that might take a lot of time, like using a service or a large database query, consider putting it into a background job . The Rails app queues up a job to send the emails, and then it can respond to the web request while the mailing is handled in the background. This has other advantages: errors calling the service won't cause an error for the user, and failed jobs due to a temporary service outage can automatically be retried.

In Rails this is done with ActiveJob and you could write a job class to send your mail.

Use ActionMailer

However, Rails also offers a class specifically for sending email in the background: ActionMailer . You can have ActionMailer use AWS with the aws-sdk-rails gem .

config.action_mailer.delivery_method = :ses