How can I attach a file from an HTTP request to an email? (Python/ AWS Lambda)

Question

I am currently attempting to migrate an email-sending microservice to the cloud. My code is in an AWS Lambda function which is triggered by an http request sent to an endpoint. If that HTTP request includes a file attachment my code should attach that file to the email it sends.

At the moment, my code sends emails fine, but when I try to send an attachment it comes out garbled. The contents of the file are changed somewhere in the process of sending it in an HTTP request and then attaching it to an email. However, .txt files can be sent with no issues. Why would it be screwing up files like that?

Thanks in advance for the help.

I suspected there was a problem with encoding so I tried (as you can see below) to send the binary data of the attachment, read that data as binary, then write an attachment object that contains that binary, but it did not help. I also tried changing how I attached the file to the HTTP request in case it was an issue with how it was being read, but there was no change. Additionally I tried uploading to an s3 bucket instead of attaching to an email, but that did not change anything either.

This is the code that sends the HTTP request:

import requests
import json
import os

url = "..."

payload = {
    "requester": "Our robotic overlords",
    "recipients": [
        "..."
    ],
    "message": "This is the message the recipient will see",
    "subject": "THIS IS A SAMPLE. DO NOT BE ALARMED"
}

headers = {
    'content-type': "multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW",
    'Accept': "application/json",
    'Cache-Control': "no-cache",
    'Postman-Token': "79f6f992-0b1b-45a4-a00e-f095be17dc56,ce0c120c-b3f6-4658-89cd-269b122f0342",
    'Host': "q4kisdfuog.execute-api.us-east-1.amazonaws.com",
    'Accept-Encoding': "gzip, deflate",
    'Connection': "keep-alive",
    'cache-control': "no-cache"
    }

toupload = open("VPC.png", "rb")

files = { 
    "file" : ("VPC.png", toupload.read(), "application/octet-stream"),
    "json" : (None, json.dumps(payload), "application/json")
}

response = requests.request("POST", url,  headers=headers, files=files)

print(response.text)

and this is the code hosted in my Lambda function that receives the HTTP request, and sends an email (with an attachment) based on it:

    # get the system time when the request is received.
    time_received = datetime.now()

    # Get the incoming request body
    en = EmailNotification("n/a", "n/a", "n/a", time_received)

    try:

        req_body = request["body"]
        first_new_line = req_body.find("\r\n")
        req_boundary = req_body[:first_new_line]
        print(req_boundary)

        req_body_parts = req_body.split(req_boundary)

        message_body = req_body_parts[len(req_body_parts)-2]
        body_start = message_body.find("{")
        message_body = message_body[body_start:]
        file_data = req_body_parts[1]

        # process the request's main body to a EmailNotification object
        req_json = json.loads(message_body)
        requester = req_json["requester"]
        recipients = req_json["recipients"]
        subject = req_json["subject"]
        message = req_json["message"]

        en.set_requester(requester)
        en.set_recipients(recipients)
        en.set_subject(subject)
        en.set_message(message)

        print("Message Good")

        file_data_parts = file_data.split("\r\n")

        file_name_start = file_data_parts[1].find("filename=") + 10
        file_name = file_data_parts[1][file_name_start : len(file_data_parts[1]) - 1]
        print(file_name)

        # add the basic info of the attached file to the EmailNotification object
        filesize = -1
        filetype_start = file_data_parts[2].find("Content-Type: ") + 14
        filetype = file_data_parts[2][filetype_start : len(file_data_parts[2])]
        print(filetype)

        attach_obj = Attachment(file_name, filetype, filesize)
        en.set_attachment(attach_obj)

        print("creates attachment")

        # Prepare the email to send 
        msg = MIMEMultipart()
        msg['From'] = EmailNotification.SERVER_EMAIL_ACCOUNT
        msg['To'] = ','.join(en.get_recipients())
        msg['Subject'] = en.get_subject()

        # prepare the main message part of the email
        main_msg = MIMEText(en.get_message(),'plain')

        print("Creates message")

        # prepare the attachment part of the email
        content_type = filetype
        main_type = content_type.split("/")[0]
        sub_type = content_type.split("/")[1]

        binary_parts = []

        for part in file_data_parts[4:]:
            binary_parts.append(BytesIO((part.join("\r\n")).encode("utf-8")))

        print("reads contents")

        path = "/tmp/" + file_name
        target_file = open(path, "wb+")

        for item in binary_parts:
            target_file.write(item.read())

        target_file.close()
        print("WRITES")

        toupload = open(path, "rb")

        att_part = MIMEBase(main_type,sub_type)
        att_part.set_payload(toupload.read())
        encoders.encode_base64(att_part)
        att_part.add_header('Content-Disposition','attachment; filename={}'.format(file_name))

        toupload.close()

        print("ACTUALLY CREATES ATTACHMENT")

        # attach each sub part to the email message
        msg.attach(main_msg)
        msg.attach(att_part)

        print("ATTACHES PARTS TO MESSAGE")

        # Send the email according to SMTP protocol
        try:
            with smtplib.SMTP(SERVER, PORT) as server:
                print(server.ehlo())
                print(server.sendmail(SERVER_EMAIL_ACCOUNT, ','.join(en.get_recipients()), msg.as_string()))','.join(en.get_recipients()), msg.as_string()))
        except Exception as ex:
            print("FAILS IN EMAIL SENDING")
            en.log()
            response = prep_response(en)
            return response
        else:
            print("SUCCEEDS")
            los = []
            for i in range(len(en.get_recipients())):
                los.append(Status.SUCCESS)
            en.set_status(los)
            en.log()
            response = prep_response(en)
            # Send a HTTP response to the client
            return response

    except Exception as ex:

        print("FAILS! w/")
        print(ex)

        # catch exception during parsing the request received
        los = []
        for i in range(len(en.get_recipients())):
            los.append(Status.REQUEST_BODY_ERROR)
        en.set_status(los)
        en.log()
        response = prep_response(en)
        return response

The attachment that is returned by this code has the correct name, and filetype, but is not openable, and has different contents when compared to the original file. For example, if I attach VPC.png (56kb) I get back VPC.png (99kb).

If I look at the actual contents of the file, the original looks like this (just a sample):

‰PNG

IHDR é ì ^ )¸ sRGB

and the attached version looks like this (just a sample):

PNG

IHDR ^ ) sRGB

Answer 1

Managed to get it myself. As I suspected, it was an encoding issue. Encoding the attachment file in Base64 before sending it, and then decoding it in the Lambda did the trick. It seems like the attachment file wasn't being read correctly before getting sent to the Lambda, which caused the issue.