Ok so I want to build an app that uses para threads. So I thought Id understand the threading first. I found this example , for the life of me I cant work out why Im not getting any results back. It doesnt throw any exceptions or anything it just runs and then finishes. Can anyone explain why the below doesnt show any results. I assumed that it was due to not using `print' or return at the end of the function but I played around with that to no avail. Helpss :) please
import urllib2
from multiprocessing.dummy import Pool as ThreadPool
urls = [
'http://www.python.org',
'http://www.python.org/about/',
'http://www.onlamp.com/pub/a/python/2003/04/17/metaclasses.html',
'http://www.python.org/doc/',
'http://www.python.org/download/',
'http://www.python.org/getit/',
'http://www.python.org/community/',
'https://wiki.python.org/moin/',
'http://planet.python.org/',
'https://wiki.python.org/moin/LocalUserGroups',
'http://www.python.org/psf/',
'http://docs.python.org/devguide/',
'http://www.python.org/community/awards/'
# etc..
]
# Make the Pool of workers
pool = ThreadPool(4)
# Open the urls in their own threads
# and return the results
results = pool.map(urllib2.urlopen, urls)
#close the pool and wait for the work to finish
pool.close()
pool.join()
To elaborate on @figs suggestion:
def opener(url):
site = urllib2.urlopen(url)
return site.read()
results = pool.map(opener, urls)
The problem is that you're not using the read()
method, which returns the html of the page.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.