Can't download AWS S3 File in nodejs

Question

I'm trying to use Amazon's S3 service, I managed to upload GZipped files to my bucket but I can't retrieve them. I tried using the code example that I've found here , everything works fine when I'm uploading the files, but I can't download them. This is my upload code:

var s3 = new AWS.S3();
s3.headBucket({Bucket: bucketName}, function (err) {
    if (err) s3.createBucket({Bucket: bucketName}, cb);
     var body = fs.createReadStream(file).pipe(zlib.createGzip());
    s3.upload({Bucket: bucketName, Key: key, Body: body}).send(cb);
});

ANd this is my download code:

 var s3 = new AWS.S3();
 var params = {Bucket: bucketName, Key: key};
 var outFile = require('fs').createWriteStream(file);
 s3.getObject(params).createReadStream().pipe(zlib.createGunzip()).pipe(outFile);

But I get error throw new Error('Cannot switch to old mode now.'); on the last line. and I can't figure out how to fix it, I'm using node 0.10.25(and I can't change it). So I tried using this:

var params = {Bucket: bucketName, Key: key};
s3.getObject(params, function(err, data) {
    var outFile = require('fs').createWriteStream(file);
    var read = AWS.util.buffer.toStream(data.Body);
    read.pipe(zlib.createGzip()).pipe(outFile);
    read.on('end', function(){cb();});
});

but often I get error 104(unexpected end of input).

Anyone has some ideas?

Answer 1

Unexpected end of input is perhaps due to pipe getting closed prematurely or some other error was encountered in the middle of reading a fixed-size block or data structure.

You can look at - https://github.com/minio/minio-js instead as well as an alternative, it is fully written in Streams2 style.

Here is an example.

$ npm install minio
$ cat >> get-object.js << EOF

var Minio = require('minio')
var fs = require('fs')

// find out your s3 end point here:
// http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region

var s3Client = new Minio({
  url: 'https://<your-s3-endpoint>',
  accessKey: 'YOUR-ACCESSKEYID',
  secretKey: 'YOUR-SECRETACCESSKEY'
})

var outFile = fs.createWriteStream('test.txt');
s3Client.getObject('mybucket', 'my-key', function(e, dataStream) {
  if (e) {
    return console.log(e)
  }
  dataStream.pipe(outFile)
})
EOF