简体   繁体   中英

Trying to Gather all tweets from the past 24hours and put them into a CSV file

Im trying to gather all the tweets from the last 24 hours and put them into a CSV file

When i do this i get

_csv.Error: iterable expected, not datetime.datetime

As an error

Can anyone help tell me how to get rid of this error and any other improvements that could possibly be made to the code

def get_all_tweets(screen_name):
# Twitter only allows access to a users most recent 3240 tweets with this method

# authorize twitter, initialize tweepy
auth = tweepy.OAuthHandler(consumer_token, consumer_secret)
auth.set_access_token(access_token, access_secret)
api = tweepy.API(auth, wait_on_rate_limit=True)

# initialize a list to hold all the tweepy Tweets
alltweets = []    

# make initial request for most recent tweets (20 is the maximum allowed count)
new_tweets = api.home_timeline (screen_name=screen_name, count=20)

# save most recent tweets
alltweets.extend(new_tweets)

# save the id of the oldest tweet less one
oldest = alltweets[-1].id - 1

page = 1
deadend = False



while len(new_tweets) > 0:
        print ("getting tweets before %s" % (oldest))

        # all subsiquent requests use the max_id param to prevent duplicates
        new_tweets = api.home_timeline(screen_name=screen_name, count=20, max_id=oldest,  page = page)

        # save most recent tweets
        alltweets.extend(new_tweets)

        # update the id of the oldest tweet less one
        oldest = alltweets[-1].id - 1

        print ("...%s tweets downloaded so far" % (len(alltweets)))

        for tweet in alltweets:

            if (datetime.datetime.now() - tweet.created_at).days < 1:

                 # transform the tweepy tweets into a 2D array that will populate the csv    
                 outtweets = [tweet.id_str, tweet.created_at, tweet.text.encode("utf-8")]

            else:
                 deadend = True
                 return
        if not deadend:
             page += 1
             time.sleep(10)

# write the csv    
with open('%s_tweetsBQ.csv' % screen_name, 'w') as f:
    writer = csv.writer(f)
    writer.writerow(["id", "created_at", "text"])
    writer.writerows(outtweets)
pass

print ("CSV written")

if __name__ == '__main__':
# pass in the username of the account you want to download
get_all_tweets("BQ")

Edit

(most recent call last):
 File "C:\Users\Barry\workspace\TwitterTest\Test1\MGo.py", line 77, in    <module>
  get_all_tweets("BQ")
File "C:\Users\Barry\workspace\TwitterTest\Test1\MGo.py", line 70, in get_all_tweets
writer.writerows(outtweets)
 _csv.Error: iterable expected, not datetime.datetime

EDIT 2

for row in outtweets:
            date_str,time_str, entries_str = row.split()
            #print(a_date,a_time, entries)
            a_time = datetime.strptime(time_str, "%H:%M:%S")
            for e in entries_str.split(','): 
                 # write the csv    
                 with open('%s_tweetsBQ.csv' % screen_name, 'w') as f:
                     writer = csv.writer(f)
                     writer.writerow(["id", "created_at", "text"])
                     writer.writerows(outtweets)
                 pass

outtweets only ever contains a single row of data. writer.writerows() expects a list of rows, that is, a list of lists:

[
  [columns, in, row, 1],
  [columns, in, row, 2],
]

You are setting outtweets like this:

outtweets = [tweet.id_str, tweet.created_at, tweet.text.encode("utf-8")]

That's just a single row. To pass this to writerows , you need to accumulate each row of data into a list, and then pass that list to writerows .

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM