Tweepy - Error 144 when populating a pandas dataframe column with tweet strings

Question

I'm populating some rows in a dataframe using the twitter ID. I've run the script the first time without the except and I had the error: [{'code': 144, 'message': 'No status found with that ID.'}] I understand that it might be because someone deleted the tweet or for other reason. However, I need to keep going!

So I used the except: pass , but it actually doesn't return anything. All the rows are empty. I've been working hard on this, but I don't know to solve it.

My dataframe:

          TweetID                text               pageType
index   
id1                     My code is not working      http://blablabla.com
id2     451864165416    Nan                         twitter
id3     849849849844    Nan                         twitter

Here is the code that doesn't return anything:

try:
    if (df['pageType'] == 'twitter').any:
        df['text'] = df.tweetID.apply(lambda x: api.get_status(x).text)
except:
    pass

That's it! Thanks a lot!

Answer 1

I'd recommend a boolean index + loc + apply :

mask = df['pageType'] == 'twitter'
df.loc[mask, 'text'] = df.loc[mask, 'twitterID']\
                           .apply(lambda x: api.get_status(x).text)

Answer 2

Problem is, your try and except setup stops execution before the apply can be completed, which in turn never creates the new column. Typically you would place this clause in a for-loop , the way you are using it. Instead you could create a custom function, so that it catches errors on tweetID values that are invalid.

def GetStuff(value):
    try:
        return api.get_status(value).text
    except:
        return "ERROR"

df['text'] = df.tweetID.apply(lambda x: GetStuff(x))

---

To meet the conditions in the comments:

Option 1

def GetStuff(value):
    try:
        return api.get_status(value).text
    except:
        return "ERROR"

df['text'] = df.where(df.tweetID == 'twitter').tweetID.apply(lambda x: GetStuff(x))

Which applies the function where tweetID == twitter , the the other values are NaN with you can replace with some other text using fillna()

Option 2

Build conditions in the GetStuff() function.

def GetStuff(value):
    if value == 'twitter':
        try:
            return api.get_status(value).text
        except:
            return "ERROR"
     else:
         return 'NotTwitter'

df['text'] = df.tweetID.apply(lambda x: GetStuff(x))