Using Streaming Data from Twitter with Meteor

Question

So far I have been able to pull down streaming real time data from Twitter. How do I use this data? I am trying to insert it into collection but I am getting this Error:

Error: Meteor code must always run within a Fiber. Try wrapping callbacks that you pass to non-Meteor libraries with Meteor.bindEnvironment.

I tried wrapping my code with a fiber but it didn't work/or I am not wrapping the right part of the code. Also, I'm not sure if this is the proper way to use streaming data in Meteor.

Posts = new Meteor.Collection('posts');

if (Meteor.isClient) {
  Meteor.call("tweets", function(error, results) {
    console.log(results); //results.data should be a JSON object
  });
}

if (Meteor.isServer) {    
  Meteor.methods({
    tweets: function(){

      Twit = new TwitMaker({
        consumer_key: '...',
        consumer_secret: '...',
        access_token: '...',
        access_token_secret: '...'
      });

      sanFrancisco = [ '-122.75', '36.8', '-121.75', '37.8' ];

      stream = Twit.stream('statuses/filter', { locations: sanFrancisco });


      stream.on('tweet', function (tweet) {
        userName = tweet.user.screen_name;
        userTweet = tweet.text;
        console.log(userName + " says: " + userTweet);
        Posts.insert({post: tweet})

      })  
    }    
  })  
}

Answer 1

Code that mutates the database needs to be run in a fiber, which is what the error is about. Code that runs in a callback from a library other than Meteor is not (necessarily) run in a fiber, so you'll need to wrap the callback function to make sure it gets run in a fiber, or at least the part of it that interacts with the database.
Meteor.bindEnvironment is not currently documented, but it is generally considered the most reliable method of wrapping callbacks. Meteor.bindEnvironment, which the error talks about, is defined here for reference: https://github.com/meteor/meteor/blob/master/packages/meteor/dynamics_nodejs.js#L63

Something like this is probably the easiest way of making this work:

tweets: function() {
  ...

  // You have to define this wrapped function inside a fiber .
  // Meteor.methods always run in a fiber, so we should be good here. 
  // If you define it inside the callback, it will error out at the first
  // line of Meteor.bindEnvironment.

  var wrappedInsert = Meteor.bindEnvironment(function(tweet) {
    Posts.insert(tweet);
  }, "Failed to insert tweet into Posts collection.");

  stream.on('tweet', function (tweet) {
    var userName = tweet.user.screen_name;
    var userTweet = tweet.text;
    console.log(userName + " says: " + userTweet);
    wrappedInsert(tweet);
  });
}

Answer 2

This works for me. Essential is to call Meteor.bindEnvironment from inside the Twit callback.

Meteor.methods({
    consumeTwitter: function () {

    var Twit = Meteor.npmRequire('twit');

    var T = new Twit({
        consumer_key:         'xxx', // API key
        consumer_secret:      'yyy', // API secret
        access_token:         'xxx',
        access_token_secret:  'xxx'
    });

    //  search twitter for all tweets containing the word 'banana'
    var now = new Date().getTime();

    var wrappedInsert = Meteor.bindEnvironment(function(tweet) {
      Tweets.insert(tweet);
    }, "Failed");


    T.get('search/tweets',
        {
            q: 'banana since:2011-11-11',
            count: 4
        },
        function(err, data, response) {
          var statuses = data['statuses'];

          for(var i in statuses) {
             wrappedInsert(statuses[i]);
          }
        }
    )}
});

Answer 3

I had written a lengthy post about Building Twitter Monitoring Apps with MeteorJS from Scratch , including the Meteor.bindEnvironment part, extract as below.

var Twit = Meteor.npmRequire(‘twit’);
var conf = JSON.parse(Assets.getText(‘twitter.json’)); 
var T = new Twit({
 consumer_key: conf.consumer.key, 
 consumer_secret: conf.consumer.secret,
 access_token: conf.access_token.key, 
 access_token_secret: conf.access_token.secret

// 
// filter the twitter public stream by the word ‘iwatch’. 
//

var stream = T.stream(‘statuses/filter’, { track: conf.keyword })

stream.on(‘tweet’, Meteor.bindEnvironment(function (tweet) {
 console.log(tweet);
 Tweets.insert(tweet);
}))

There are only two functions added:

Meteor.bindEnvironment()

This function helps us to bind a function to the current value of all the environment variables.

Have fun!