[英]Multi-threading in python with loop
我正在嘗試使用python中的多線程技術解決項目Euler中的問題8 。
在1000位數字中找到五個連續數字的最大乘積。 該號碼可以在這里找到。
我的方法是從原始列表的5個塊中生成乘積,並重復此過程5次,每次起始索引向右移動一個。
這是我的線程類
class pThread(threading.Thread):
def __init__(self, l):
threading.Thread.__init__(self)
self.l = l
self.p = 0
def run(self):
def greatest_product(l):
"""
Divide the list into chunks of 5 and find the greatest product
"""
def product(seq):
return reduce(lambda x,y : x*y, seq)
def chunk_product(l, n=5):
for i in range(0, len(l), n):
yield product(l[i:i+n])
result = 0
for p in chunk_product(num):
result = result > p and result or p
return result
self.p = greatest_product(self.l)
當我嘗試創建5個線程來覆蓋原始列表中的所有5位數字塊時,下面的手動方法給出了正確的答案, num
是我從文本中解析出的一位數字的列表:
thread1 = pThread(num)
del num[0]
thread2 = pThread(num)
del num[0]
thread3 = pThread(num)
del num[0]
thread4 = pThread(num)
del num[0]
thread5 = pThread(num)
thread1.start()
thread2.start()
thread3.start()
thread4.start()
thread5.start()
thread1.join()
thread2.join()
thread3.join()
thread4.join()
thread5.join()
def max(*args):
result = 0
for i in args:
result = i > result and i or result
return result
print max(thread1.p, thread2.p, thread3.p, thread4.p, thread5.p)
但這不能給出正確的結果:
threads = []
for i in range(0, 4):
tmp = num[:]
del tmp[0:i+1]
thread = pThread(tmp)
thread.start()
threads.append(thread)
for i in range(0, 4):
threads[i].join()
我在這里做錯了什么? 我對多線程非常陌生,因此請保持謹慎。
有3個問題:
首先是“手動”方法不能給出正確的答案。 碰巧問題的正確答案是在列表開頭的偏移量4處。 您可以使用以下方法查看:
import operator as op print max(reduce(op.mul, num[i:i+5]) for i in range(1000)) for k in range(5): print max(reduce(op.mul, num[i:i+5]) for i in range(k, 1000, 5))
您的“手動”方法的一個問題是線程共享num
變量,每個線程具有相同的列表。 因此,當您執行del num[0]
,所有threadX.l
受到影響。 您始終得到相同答案的事實是由於第二個問題。
線
for p in chunk_product(num):
應該:
for p in chunk_product(l):
因為您要使用功能greatest_product(l)
的參數,而不要使用全局變量num
。
在第二種方法中,由於循環范圍超過[0, 1, 2, 3]
僅產生4個線程。 另外,您要刪除值tmp[0:i]
而不是tmp[0:i+1]
。 這是代碼:
threads = [] for i in range(5): tmp = num[:] del tmp[0:i] thread = pThread(tmp) thread.start() threads.append(thread) for i in range(5): threads[i].join() print len(threads), map(lambda th: th.p, threads) print max(map(lambda th: th.p, threads))
我主要嘗試了一些多處理實踐,並學習了如何使用argparse。
萬一您的機器沒有很多東西,這大約需要4-5個演出。
python euler.py -l 50000000 -n 100 -p 8
Took 5.836833333969116 minutes
The largest product of 100 consecutive numbers is: a very large number
如果在命令行中鍵入python euler.py -h,則會得到:
usage: euler.py [-h] -l L [L ...] -n N [-p P]
Calculates the product of consecutive numbers and return the largest product.
optional arguments:
-h, --help show this help message and exit
-l L [L ...] A single number or list of numbers, where each # is seperated
by a space
-n N A number that specifies how many consecutive numbers should be
multiplied together.
-p P Number of processes to create. Optional, defaults to the # of
cores on the pc.
和代碼:
"""A multiprocess iplementation for calculation the maximum product of N consecutive
numbers in a given range (list of numbers)."""
import multiprocessing
import math
import time
import operator
from functools import reduce
import argparse
def euler8(alist,lenNums):
"""Returns the largest product of N consecutive numbers in a given range"""
return max(reduce(operator.mul, alist[i:i+lenNums]) for i in range(len(alist)))
def split_list_multi(listOfNumbers,numLength,threads):
"""Split a list into N parts where N is the # of processes."""
fullLength = len(listOfNumbers)
single = math.floor(fullLength/threads)
results = {}
counter = 0
while counter < threads:
if counter == (threads-1):
temp = listOfNumbers[single*counter::]
if counter == 0:
results[str(counter)] = listOfNumbers[single*counter::]
else:
prevListIndex = results[str(counter-1)][-int('{}'.format(numLength-1))::]
newlist = prevListIndex + temp
results[str(counter)] = newlist
else:
temp = listOfNumbers[single*counter:single*(counter+1)]
if counter == 0:
newlist = temp
else:
prevListIndex = results[str(counter-1)][-int('{}'.format(numLength-1))::]
newlist = prevListIndex + temp
results[str(counter)] = newlist
counter += 1
return results,threads
def worker(listNumbers,number,output):
"""A worker. Used to run seperate processes and put the results in the queue"""
result = euler8(listNumbers,number)
output.put(result)
def main(listOfNums,lengthNumbers,numCores=multiprocessing.cpu_count()):
"""Runs the module.
listOfNums must be a list of ints, or single int
lengthNumbers is N (an int) where N is the # of consecutive numbers to multiply together
numCores (an int) defaults to however many the cpu has, can specify a number if you choose."""
if isinstance(listOfNums,list):
if len(listOfNums) == 1:
valuesToSplit = [i for i in range(int(listOfNums[0]))]
else:
valuesToSplit = [int(i) for i in listOfNums]
elif isinstance(listOfNums,int):
valuesToSplit = [i for i in range(listOfNums)]
else:
print('First arg must be a number or a list of numbers')
split = split_list_multi(valuesToSplit,lengthNumbers,numCores)
done_queue = multiprocessing.Queue()
jobs = []
startTime = time.time()
for num in range(split[1]):
numChunks = split[0][str(num)]
thread = multiprocessing.Process(target=worker, args=(numChunks,lengthNumbers,done_queue))
jobs.append(thread)
thread.start()
resultlist = []
for i in range(split[1]):
resultlist.append(done_queue.get())
for j in jobs:
j.join()
resultlist = max(resultlist)
endTime = time.time()
totalTime = (endTime-startTime)/60
print("Took {} minutes".format(totalTime))
return print("The largest product of {} consecutive numbers is: {}".format(lengthNumbers, resultlist))
if __name__ == '__main__':
#To call the module from the commandline with arguments
parser = argparse.ArgumentParser(description="""Calculates the product of consecutive numbers \
and return the largest product.""")
parser.add_argument('-l', nargs='+', required=True,
help='A single number or list of numbers, where each # is seperated by a space')
parser.add_argument('-n', required=True, type=int,
help = 'A number that specifies how many consecutive numbers should be \
multiplied together.')
parser.add_argument('-p', default=multiprocessing.cpu_count(), type=int,
help='Number of processes to create. Optional, defaults to the # of cores on the pc.')
args = parser.parse_args()
main(args.l, args.n, args.p)
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.