我正在尝试编写一个 python 脚本来使用 Mapreducer 读取 csv 并且我收到错误 ValueError: too many values to unpack (expected 2)

Question

0

I'm running the following Python code in MapReduce:

from mrjob.job import MRJob
from mrjob.step import MRStep


class productRevenue(MRJob):
    #each input lines consists of product, productCategory, price, and paymentMode
    def mapper_get_product(self, _, line):
        # create a key-value pair with key: product and value: price
        line_cols = line.split(',')
        yield line_cols[1], float(line_cols[2])

    def combiner_count_product(self, product, counts):
        # consolidates all key-value pairs of mapper function (performed at mapper nodes)
        yield product, sum(counts)

    def reducer_count_product(self, product, counts):
        # final consolidation of key-value pairs at reducer nodes
        yield None, '${:,.2f}'.format(sum(counts), product)

    def reducer_find_max(self, _, product_count_pairs):
        yield max(product_count_pairs)

    def steps(self):
        return [
            MRStep(mapper=self.mapper_get_product,
                   combiner=self.combiner_count_product,
                   reducer=self.reducer_count_product),
            MRStep(reducer=self.reducer_find_max)
        ]       

if __name__ == '__main__':
    productRevenue.run()

I get the error: ValueError: too many values to unpack (expected 2) but I can't figure out why. Any suggestions? The cmd line code I'm running is: python test.py data.csv The .CSV can be download here https://users.cs.fiu.edu/~prabakar/database/4722sp19/abarr054-3495916/

Answer 1

In reducer_find_max() function, you are passing one parameter in yield. The yield function basically outputs a key and a value.

Please pass two parameters as "yield max(product_count_pairs), None"