如何通过urllib2.urlopen下载文件

Question

i know how to download simple file from urrlib2.urlopen

but my end URL is not simple it has special character in it :

" www.math.ualberta.ca/mss/misc/A Mathematician's Apology.pdf "

special character Mathematician's ' is in this path

some how i know that

http://www.math.ualberta.ca/mss/misc/A%20Mathematician%27s%20Apology.pdf

is url i have to use to download file but if i don't have this kind of end URL with me every time

please give me solution so that i can download file which has special URL

i have basic method that can be used but i don't know how to use this

1. urllib.quote(string[, safe])
2. urllib.quote_plus(string[, safe])
3. urllib.unquote(string)
4. urllib.unquote_plus(string)

please help me with this method by example

thank you

Answer 1

Why not use something like this?

filename = url.split('/')[-1]
cleanurl = urllib.quote(url)
urllib.urlretrieve(cleanurl, filename)

Answer 2

You want to quote only the path component of the URL, not the whole thing.

The cleanest way to do this is to split it into pieces with urlparse , quote the path component, and rejoin the whole thing.

But as it turns out, urlparse automatically quotes the path anyway. This isn't really documented, but it's been true for every version so far, so if you're willing to rely on that, it's as simple as this:

>>> url = "www.math.ualberta.ca/mss/misc/A Mathematician's Apology.pdf"
>>> url = urlparse.urlparse(url).geturl()
>>> url
'http://www.math.ualberta.ca/mss/misc/A%20Mathematician%27s%20Apology.pdf'

---

If you actually just have a host and path, you can in fact just use urllib.quote . With a full URL, that will quote the : character between the scheme and host, but if you don't have a scheme, as in you example, that's not a problem. (Of course it will quote the stray spaces in your example too… but those are going to be a problem no matter what you do, so your first step has to be removing them.)

>>> url = " www.math.ualberta.ca/mss/misc/A Mathematician's Apology.pdf "
>>> url = urllib.quote(url.strip())
>>> url
'www.math.ualberta.ca/mss/misc/A%20Mathematician%27s%20Apology.pdf'

You'll still need to add a scheme before this is actually useful, of course.