简体   繁体   中英

Threading using pool and map function

Ok so I want to build an app that uses para threads. So I thought Id understand the threading first. I found this example , for the life of me I cant work out why Im not getting any results back. It doesnt throw any exceptions or anything it just runs and then finishes. Can anyone explain why the below doesnt show any results. I assumed that it was due to not using `print' or return at the end of the function but I played around with that to no avail. Helpss :) please

import urllib2 
from multiprocessing.dummy import Pool as ThreadPool 

urls = [
    'http://www.python.org', 
    'http://www.python.org/about/',
    'http://www.onlamp.com/pub/a/python/2003/04/17/metaclasses.html',
    'http://www.python.org/doc/',
    'http://www.python.org/download/',
    'http://www.python.org/getit/',
    'http://www.python.org/community/',
    'https://wiki.python.org/moin/',
    'http://planet.python.org/',
    'https://wiki.python.org/moin/LocalUserGroups',
    'http://www.python.org/psf/',
    'http://docs.python.org/devguide/',
    'http://www.python.org/community/awards/'
    # etc.. 
]

# Make the Pool of workers
pool = ThreadPool(4) 
# Open the urls in their own threads
# and return the results
results = pool.map(urllib2.urlopen, urls)
#close the pool and wait for the work to finish 
pool.close() 
pool.join() 

To elaborate on @figs suggestion:

def opener(url):
    site = urllib2.urlopen(url)
    return site.read()


results = pool.map(opener, urls)

The problem is that you're not using the read() method, which returns the html of the page.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM