Random -1 response status while uploading to AWS S3 using pre-signed urls

Question

Our front-end side uploads document to S3 using pre-signed urls and seems to be failing randomly. This part of functionality if very critical to us.

Our pre-signed urls are generated by back-end using boto3.

[...]
@classmethod
def get_presigned_url(cls, filename, user, content_type, size=None):
    client = cls.get_s3_client()
    import logging
    logging.info(cls.generate_keyname(filename, user))
    key = cls.generate_keyname(filename, user)
    params ={'Bucket': cls.s3_staging_bucket, 'Key': key,
             "ContentType": content_type}
    if size:
        params['ContentLength'] = size

    # It's private as default
    if cls.is_private:
        params['ACL'] = 'private'
    else:
        params['ACL'] = 'public-read'

    return client.generate_presigned_url(
        'put_object',
        Params=params,
        ExpiresIn=600
    ), cls.get_url(key, cls.s3_staging_bucket)
[...]

So the front-end sends following information to request upload link:

        [...]
        // Request Presigned url
        Restangular.all('upload').all('get_presigned_url').post(
            {
                'resource_type': 'candidate-cv',
                'filename': vm.file.name,
                'size': vm.file.size || null,
                'content_type': vm.file.type || 'application/octet-stream'
            }
        ).then(
        [...]

Things to note in above example: the size and type are not available in all browsers so I have to fallback to defaults.

Once link is retrieved front-end attempts to upload directly to s3 bucket:

                [...]
                $http.put(
                    data['presigned_url'],
                    vm.file,
                    {
                        headers: {
                            'Content-Type':  vm.file.type || 'application/octet-stream',
                            'Authorization': undefined // Needed to remove default ApiKey
                        }
                    }
                ).then(
                [...]

The above code gives sometimes -1 response. Sometimes is a problem because it happens a way to often. Probably something around 3% of cases.

We have checked inserted debug logger that sends debug information on every bad response but everything really seems to be alright there.

Our facts so far:

• It seemed to me in the beginning that's connectivity issue but should the response status be 0 instead -1?
• It happens way too often for connectivity issue (~3%)
• It happens on whole range or user agents Windows/Mac Chrome/Edge Mobile/Desk old and new.
• It happens with whole range of document formats docx/doc/pdf.
• Same users tried several times in a row during 1 hour period all failed with -1.
• Same users with same user-agents seem to be able to do upload successfully day before or day after.
• We are unable replicate it.

What do we do wrong? What direction should we take to investigate this problem? What next steps should we follow to solve the issue?

Thanks for your input.

EDIT:

As @tcrite suggested that -1 means client side timeout. That seem to be correct to replicate the problem in my local env. We updated production server adding long client timeouts: 250 seconds.

But just recently we have got several -1 responses. The user tried to submit file 6 times in 2 minutes with all resulting with -1 response code and timeout config was present:

Response:

{  
   "data":null,
   "status":-1,
   "config":{  
      "method":"PUT",
      "transformRequest":[  
         null
      ],
      "transformResponse":[  
         null
      ],
      "jsonpCallbackParam":"callback",
      "headers":{  
         "Content-Type":"application/msword",
         "Accept":"application/json, text/plain, */*"
      },
      "timeout":250000,
      "url":"https://stackoverflow-question.s3.amazonaws.com/uploads/files/a-b-a36b9b2f216..."
   }
}

It can't be S3 timeout as I tried in my local env to upload file on slow connection for ~5 minutes and it was uploaded sucessfully.

Answer 1

I think you should make a server side web application to upload files ( rather than browse based angular ).Because browser are sometime restricted by company policy. Check this python django application.I believe youa re already using python.

[https://testdriven.io/blog/storing-django-static-and-media-files-on-amazon-s3/][1]