Node.js Twitter API游标

Question

I'm using npm-twit to get followers of a specific account.

The Twitter API returns up to 5000 results from a single GET request.

If the user I'm querying has over 5000 followers a "next_cursor" value is returned with the data.

To get the next 5000 results, I need to re-run the GET function, passing it the "next_cursor" value as an argument. I just can't seem to work out how to do it.

I was thinking a while loop, but I can't reset the global variable, I think because of scope:

var cursor = -1

while ( cursor != 0 ) { 

  T.get('followers/ids', { screen_name: 'twitter' },  function (err, data, response) {

  // Do stuff here to write data to a file

  cursor = data["next_cursor"];

  })

}

Obviously I'm not a JS genius, so any help would be much appreciated.

Answer 1

The issue you are having is due to Node.js being asynchronous .

T.get('followers/ids', { screen_name: 'twitter' },  function getData(err, data, response) {

  // Do stuff here to write data to a file

  if(data['next_cursor'] > 0) T.get('followers/ids', { screen_name: 'twitter', next_cursor: data['next_cursor'] }, getData);

  })

}

Please note:

1. I gave a name to the internal callback function. That is so that we can recursively call it from the inside.
2. The loop is replaced with a recursive callback.
3. If there is a next_cursor data, then we call T.get using the same function getData .

Be aware that Do stuff here code will be executed many times (as many as there are next cursors). Since it is recursive callback - the order is guaranteed.

---

If you do not like the idea of recursive callbacks, you can avoid it by:

1. Finding out beforehand all the next_cursor 's if possible, and generate requests using for loop.
2. Alternatively, use asynchronous-helper modules like Async (though for learning purposes, I would avoid modules unless you are fluent in the concept already).

Answer 2

Consider testing with some 5K+ account.

    const T = new Twit(tokens)

    function getFollowers (screenName, followers = [], cur = -1) {
      return new Promise((resolve, reject) => {
        T.get('followers/ids', { screen_name: screenName, cursor: cur, count: 5000 }, (err, data, response) => {
          if (err) {
            cur = -1
            reject(err)
          } else {
            cur = data.next_cursor
            followers.push(data.ids)
            if (cur > 0) {
              return resolve(getFollowers(screenName, followers, cur))
            } else {
              return resolve([].concat(...followers))
            }
          }
        })
      })
    }

    async function getXaqron () {
      let result = await getFollowers('xaqron')
      return result
    }

 console.log(getXaqron().catch((err) => {
  console.log(err) // Rate limit exceeded
}))

Answer 3

Struggled with this one.. Everything seemed to work, but data['next_cursor'] didn't change, EVER!

Code should be like this:

T.get('followers/ids', { screen_name: 'twitter' },  function getData(err, data, response) {

  // Do stuff here to write data to a file

  if(data['next_cursor'] > 0) T.get('followers/ids', { screen_name: 'twitter', cursor: data['next_cursor'] }, getData);

  })

}

Parameter for Twit isn't "next_cursor", it's just "cursor" ;)