使用 AWS SDK (S3.putObject) 上传一个 Readable stream 到 S3 (node.js)

Question

My goal is to upload a Readable stream to S3.

The problem is that AWS api seems to accept only a ReadStream as a stream argument.

For instance, the following snippet works just fine:

const readStream = fs.createReadStream("./file.txt") // a ReadStream

await s3.putObject({
    Bucket: this.bucket,
    Key: this.key,
    Body: readStream,
    ACL: "bucket-owner-full-control"
}

Problem starts when I try to do the same with a Readable ( ReadStream extends stream.Readable ).

The following snippet fails

const { Readable } = require("stream")
const readable = Readable.from("data data data data") // a Readable

await s3.putObject({
    Bucket: this.bucket,
    Key: this.key,
    Body: readable,
    ACL: "bucket-owner-full-control"
}

the error I get from AWS sdk is: NotImplemented: A header you provided implies functionality that is not implemented

*** Note that I prefer a Readable rather than a ReadStream since I'd like to allow passing streams that are not necessarily originated from a file - an in-memory string for instance. So a possible solution could be converting a Readable to a Readstream to work with the SDK.

Any help will be much appreciated!

Answer 1

Given the Readable is no longer a file that has metadata such as MIME type and content length you'd need to update your putObject to include those values:

const { Readable } = require("stream")
const readable = Readable.from("data data data data") // a Readable

await s3.putObject({
    Bucket: this.bucket,
    Key: this.key,
    Body: readable,
    ACL: "bucket-owner-full-control",
    ContentType: "text/plain",
    ContentLength: 42 // calculate length of buffer
}

Hopefully that helps!