0
I'm running the following Python code in MapReduce:
from mrjob.job import MRJob
from mrjob.step import MRStep
class productRevenue(MRJob):
#each input lines consists of product, productCategory, price, and paymentMode
def mapper_get_product(self, _, line):
# create a key-value pair with key: product and value: price
line_cols = line.split(',')
yield line_cols[1], float(line_cols[2])
def combiner_count_product(self, product, counts):
# consolidates all key-value pairs of mapper function (performed at mapper nodes)
yield product, sum(counts)
def reducer_count_product(self, product, counts):
# final consolidation of key-value pairs at reducer nodes
yield None, '${:,.2f}'.format(sum(counts), product)
def reducer_find_max(self, _, product_count_pairs):
yield max(product_count_pairs)
def steps(self):
return [
MRStep(mapper=self.mapper_get_product,
combiner=self.combiner_count_product,
reducer=self.reducer_count_product),
MRStep(reducer=self.reducer_find_max)
]
if __name__ == '__main__':
productRevenue.run()
I get the error: ValueError: too many values to unpack (expected 2) but I can't figure out why. Any suggestions? The cmd line code I'm running is: python test.py data.csv The .CSV can be download here https://users.cs.fiu.edu/~prabakar/database/4722sp19/abarr054-3495916/
In reducer_find_max() function, you are passing one parameter in yield. The yield function basically outputs a key and a value.
Please pass two parameters as "yield max(product_count_pairs), None"
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.