Django - Larger files failing to upload to heroku

Question

I am having trouble doing a post to a django app hosted on heroku. I have a form that submits 3 images and everything works fine when I use smaller images of about 100kb, however when I user larger images of ~3MB the upload fails with error in heroku logs showing as at=error code=H13 desc="Connection closed without response" method=POST path="/"

In django, am simply saving the images then emailing them as shown below, where formdata is holding the images. Hope the snippet is enough:

for each in form_data:
    pic = form_data[each]
    if pic:
        filename = os.path.join(self.location,f"{i} - {pic.name}")
        imgbytes = pic.read()
        with open(filename, 'wb+') as destination:
            destination.write(imgbytes)

        i+=1
        fileholder.append(filename)
email = EmailMessage(
        subject = 'Hello',
        body = 'Body goes here',
        from_email = 'example@yahoo.com',
        to = ['test@google.com'],
        )

for each in fileholder:
    email.attach_file(each)
email.attach_file(newpath)
email.send()

What is causing this and how can i ensure any image size is uploaded succesfully?

Answer 1

Ok so I found out all requests on heroku will timeout after 30sec which is what was happening in my case. The uploads where taking longer than that and my solution was to process the uploads client side and upload directly to S3 instead.

More info here

Answer 2

Django has a settings DATA_UPLOAD_MAX_MEMORY_SIZE which prevent file larger than 2.5MB (Default: 2621440 (ie 2.5 MB)) to be uploaded and raises a SuspiciousOperation (RequestDataTooBig) Exception

Try changing it in your settings.py

Answer 3

As my know,almost all web-frameworks will limit the data's size in one request. such as Django , Tornado , Spring .

And it is not safe,if not limit the max size of one request.

As the MSR974 say,change the DATA_UPLOAD_MAX_MEMORY_SIZE ,it is a way to change the setting about the max size of one request.But in your question:ensure any image size is uploaded succesfully.Key words is any size

Have a try on multipart upload?

Which is by the way of spliting one file into two or more files and merging them with order after upload to backend.(hope I explaned it clearly)