Python从URL读取JSON

Question

With Python 3, I want to read a JSON from url. This is what I got:

import json
import urllib.request
url_address = 'https://api.twitter.com/1.1/statuses/user_timeline.json'
with urllib.request.urlopen(url_address) as url:
    data = json.loads(url.read())
print(data)

However, it gives: HTTPError: HTTP Error 400: Bad Request

Answer 1

To me below code is working- in Python 2.7

import json
from urllib import urlopen
url_address = ['https://api.twitter.com/1.1/statuses/user_timeline.json']
for i in url_address:
    resp = urlopen(i)
    data = json.loads(resp.read())
print(data)

Output-

{u'errors': [{u'message': u'Bad Authentication data.', u'code': 215}]}

If you use requests module-

import requests
url_address = ['https://api.twitter.com/1.1/statuses/user_timeline.json']
for i in url_address:
    resp = requests.get(i).json()
    print resp

Output-

{u'errors': [{u'message': u'Bad Authentication data.', u'code': 215}]}

Answer 2

twitter is returning 400 code(bad request) so exception is firing. you can use try...except in your code sample and read exception body to get response json {"errors":[{"code":215,"message":"Bad Authentication data."}]}

import json
import urllib.request
from urllib.error import HTTPError


try:

    url_address ='https://api.twitter.com/1.1/statuses/user_timeline.json'
    with urllib.request.urlopen(url_address) as url:
        data = json.loads(url.read())
        print(data)

except HTTPError as ex:
    print(ex.read())

Answer 3

I prefer aiohttp , it's asynchronous. Last time I checked, urllib was blocking. This means that it will prevent other parts of the script from running while it itself is running. Here's an example for aiohttp :

import aiohttp
async with aiohttp.ClientSession() as session:
async with session.get('https://api.github.com/users/mralexgray/repos') as resp:

    print(await resp.json())