[英]MRJob :- Display intermediate values in map reduce
在使用python MRJob庫運行mapreduce程序時,如何在終端上顯示中間值(即打印變量或列表)?
您可以使用sys.stderr.write()將結果輸出為標准錯誤。 這是一個例子:
from mrjob.job import MRJob
import sys
class MRWordCounter(MRJob):
def mapper(self, key, line):
sys.stderr.write("MAPPER INPUT: ({0},{1})\n".format(key,line))
for word in line.split():
yield word, 1
def reducer(self, word, occurrences):
occurencesList= list(occurrences)
sys.stderr.write("REDUCER INPUT: ({0},{1})\n".format(word,occurencesList))
yield word, sum(occurencesList)
if __name__ == '__main__':
MRWordCounter.run()
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.