来自浏览器JS SDK的AWS S3 multipart / uploadPart静默失败

Question

I'm trying a multipart s3 upload from the browser with the JS SDK. I have no trouble with createMultipartUpload, but I get no data back from uploadPart. I can't call completeMultipartUpload because I don't get any eTags back. I get the $response part of the object only, which indicates a 200 status and that all the parameters I passed were defined and the proper datatypes. I can't see any of the parts in my bucket, although I don't know if they're going to a special "parts" place that I can't access.

Here's my code:

    const createParams = {
      Bucket,
      Key: `${uuid()}.${getExtension(file.type)}`,
      ContentType: file.type,
      ContentDisposition: 'attachment'
    }
    return s3.createMultipartUpload(createParams).promise()
    .then(result => {
      console.log(result);
      console.log('chunking...')
      let chunkArr = chunker(file);
      let chunkMap = Promise.map(chunkArr, (chunk, index) => {
        const chunkParams = {
          Body: chunk,
          Bucket: result.Bucket,
          Key: result.Key,
          PartNumber: index + 1,
          UploadId: result.UploadId,
          ContentLength: chunk.size
        }
        console.log(chunkParams)
        return s3.uploadPart(chunkParams).promise();
      });
      return Promise.all(chunkMap);
    })
    .then(result => {
      console.log(result);
      return Promise.resolve(true)
      // let stopParams = {
      //
      // }
      // return s3.completeMultipartUpload(stopParams).promise();
    })
    .catch(err => {
      throw err;
      return Promise.reject(err);
    });

s3 instance looks like this:

import AWS from 'aws-sdk';
AWS.config.setPromisesDependency(Promise);

const s3 = new AWS.S3({
    apiVersion: '2006-03-01',
    accessKeyId: credentials.credentials.AccessKeyId,
    secretAccessKey: credentials.credentials.SecretAccessKey,
    sessionToken: credentials.credentials.SessionToken,
    sslEnabled: true,
    s3ForcePathStyle: true,
    httpOptions: {
      xhrAsync: true,
      xhrWithCredentials: true
    }
  })

chunker function looks like this:

  const chunkFile = (file) => {
    console.log(typeof(file));
    const fileSize = file.size;
    const chunkSize = 5242881; // bytes
    let offset = 0;
    let chunkArr = [];

    const chunkReaderBlock = (_offset, _file) => {
      console.log(_offset);

      if (_offset >= fileSize) {
        console.log("Done reading file");
        return chunkArr;
      }

      let blob = _file.slice(_offset, chunkSize + _offset);
      console.log(blob);
      console.log(typeof(blob));
      chunkArr.push(blob);
      return chunkReaderBlock(chunkSize + _offset, _file);
    }

    return chunkReaderBlock(offset, file);
  }

The response object I'm getting back looks like this:

    (2)[{…}, {…}] 

0: {
  $response: Response
}

1: $response: Response 
  cfId: undefined 
  data: {
    $response: Response
  }
  error: null 
  extendedRequestId: undefined 
  httpResponse: HttpResponse 
    body: Uint8Array[] 
    headers: {}
    statusCode: 200 
    statusMessage: "OK"
    stream: EventEmitter {
      _events: {…},
      _maxListeners: undefined,
      statusCode: 200,
      headers: {…}
    }
    streaming: false 
    _abortCallback: ƒ callNextListener(err) __proto__: Object 
    maxRedirects: 10 
    maxRetries: 3 
    redirectCount: 0 
    request: Request {
      domain: undefined,
      service: f… s.constructor,
      operation: "uploadPart",
      params: {…},
      httpRequest: HttpRequest,
      …
    }
  retryCount: 0 
  __proto__: Object 
  __proto__: Object 
 length: 2 
__proto__: Array(0)

Any ideas? This is in React and my test file is 9.xx MB. I also tried with callbacks, and uploading one part at a time, and got the same thing.

Answer 1

In a cross-origin context, you'd need this in your bucket's CORS configuration:

<ExposeHeader>ETag</ExposeHeader>

ExposeHeader — Identifies the response headers ... that customers will be able to access from their applications (for example, from a JavaScript XMLHttpRequest object)."

https://docs.aws.amazon.com/AmazonS3/latest/dev/cors.html

To clarify what's going in here, CORS isn't an access restriction mechanism -- it's a mechanism for giving the browser permission to do something that it otherwise assumes might not be something the user would want to happen. It tells the browser to give JavaScript permission to do and see things that would not otherwise be allowed.

From the Mozilla CORS documentation:

By default, only the 6 simple response headers are exposed:

Cache-Control Content-Language Content-Type Expires Last-Modified Pragma

If you want clients to be able to access other headers, you have to list them using the Access-Control-Expose-Headers header.

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Expose-Headers

In S3, the way you set the Access-Control-Expose-Headers response header is by configuring <ExposeHeaders> (above). Otherwise, JavaScript can't see them.

---

I can't see any of the parts in my bucket, although I don't know if they're going to a special "parts" place that I can't access.

They are. Use the listMultipartUploads to find abandoned uploads, and abortMultipartUploads to delete partial uploads and free the allocated storage space for the parts you uploaded. Otherwise, uploads you don't complete will linger indefinitely and you'll be billed for storage of the parts. Also, you can create a bucket lifecycle policy to dispose of them automatically after so many days -- almost always a good idea.