How do I store large files using Meteor?

Question

I'm building a Meteor (meteorjs) app that needs to store and display PDF files, sometimes as large as 500Mb. GridFS doesn't seem to be integrated yet so I'm wondering if it's worth using Meteor in this case or stick to Rails.

Ideally I would not use S3 - I'd like to keep the files on my server.

UPDATE: it seems it's possible to connect outside of Meteor directly, I don't need PDFs to be automatically moved - and it likely doesn't make sense.

More specifically I'm now looking at: MongoDB -> ElasticSearch using https://github.com/richardwilly98/elasticsearch-river-mongodb

Using the instructions at https://github.com/richardwilly98/elasticsearch-river-mongodb/wiki

Answer 1

You can use GridFS inside meteor without touching any extra package

var db = MongoInternals.defaultRemoteCollectionDriver().mongo.db; //grab the database object
var GridStore = MongoInternals.NpmModule.GridStore;


WebApp.connectHandlers.use('/someurl', function(req, res) {
  var bigFile = new GridStore(db, 'bigfile.iso', 'r') //to read
  bigFile.open(function(error, result) {

    if (error) return

    bigFile.stream(); //stream the file
    bigFile.on('error', function(e) {...}) //handle error etc
    bigFile.on('end', function() {bigFile.close();}); //close the file when done

    bigFile.pipe(res); //pipe the file to res
  });
});

However, the current GridStore/mongo (v1.3.x) used by Meteor is a bit dated, the newest verion is 2.x from http://mongodb.github.io/node-mongodb-native/2.0/api-docs/ The v1.x doesnt seem to pipe well so you may need to use the newer version

The second option

var db = MongoInternals.defaultRemoteCollectionDriver().mongo.db; //grab the database object
var GridStore = Npm.require('mongodb').GridStore; //add Npm.depends({mongodb:'2.0.13'}) in your package.js

WebApp.connectHandlers.use('/someurl', function(req, res) {
    var bigFile = new GridStore(db, 'bigfile.iso', 'r').stream(true); //the new API doens't require bigFile.open() and will close automatically on end
    bigFile.on('error', function(e) {...}); //handle error etc
    bigFile.on('end', function() {...});
    bigFile.pipe(res); //pipe the file to res
});

In this example, I use the WebApp.connectHandlers, but of course you can use iron: router or something. I tried with a file of 500 MB and it pipes all well. You also need to set the res.writeHead(200) and other stuff such as content-type, etc