在python中测试用户代理欺骗

Question

I'm new to python and using python 3. I'm trying to download a webpage and what I want to know is if there is a way to actually see the name of the user agent the way a system admin or google sees it. In my code I download and save the webpage to a text file like this :

#Import 
from urllib.request import urlopen,Request

url1 = urlopen(Request(url,  headers={'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.47 Safari/537.36'}))

 #Create file and write
 f=open('mah_a.txt','wb')
 f.write(url1.read())
 f.close()

How do I check to see if my user agent name has changed ?

Answer 1

You changed the User-Agent header, yes. You can use a online echo server like httpbin.org if you want to see what a server received:

url = 'http://httpbin.org/get'
url1 = urlopen(Request(url,  headers={'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.47 Safari/537.36'}))
print(url1.read().decode('utf8'))

Demo:

>>> from urllib.request import urlopen,Request
>>> url = 'http://httpbin.org/get'
>>> url1 = urlopen(Request(url,  headers={'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.47 Safari/537.36'}))
>>> print(url1.read().decode('utf8'))
{
  "args": {}, 
  "headers": {
    "Accept-Encoding": "identity", 
    "Host": "httpbin.org", 
    "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.47 Safari/537.36"
  }, 
  "origin": "188.29.165.166", 
  "url": "http://httpbin.org/get"
}