I was trying to mine twitter data using tweepy and the data loaded into a JSON file , however the following code :
import tweepy
from tweepy import OAuthHandler
import json
consumer_key = '****'
consumer_secret = '****'
access_token = '****'
access_secret = '****'
auth = OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_secret)
api = tweepy.API(auth)
f = open('twitterdata.json', 'a+')
for status in tweepy.Cursor(api.home_timeline).items(10):
json.dump(status._json, f)
line = f.readline()
tweet = json.loads(line)
print json.dumps(tweet, indent = 4)
produces the error :
Traceback (most recent call last):
File "mytwittermine.py", line 21, in <module>
tweet = json.loads(line)
File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
UPDATE
As suggested in one of the answers I now add a newline in every iteration of the for
loop , so the code now is :
import tweepy
from tweepy import OAuthHandler
import json
consumer_key = '****'
consumer_secret = '****'
access_token = '****'
access_secret = '****'
auth = OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_secret)
api = tweepy.API(auth)
f = open('twitterdata.json', 'a+')
for status in tweepy.Cursor(api.home_timeline).items(10):
json.dump(status._json, f)
f.write('\n')
f.seek(0)
line = f.readline()
tweet = json.loads(line)
print json.dumps(tweet, indent = 4)
The above code gives the ValueError:
Traceback (most recent call last):
File "mytwittermine.py", line 23, in <module>
tweet = json.loads(line)
File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 367, in decode
raise ValueError(errmsg("Extra data", s, end, len(s)))
ValueError: Extra data: line 1 column 3115 - line 2 column 1 (char 3114 - 301245)
You write to a file, then read withour rewinding, so nothing gets read. Add f.seek(0)
between writing and reading.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.