我正在尝试编写一个 python 脚本来使用 Mapreducer 读取 csv 并且我收到错误 ValueError: too many values to unpack (expected 2)

Question

0

我在 MapReduce 中运行以下 Python 代码：

from mrjob.job import MRJob
from mrjob.step import MRStep


class productRevenue(MRJob):
    #each input lines consists of product, productCategory, price, and paymentMode
    def mapper_get_product(self, _, line):
        # create a key-value pair with key: product and value: price
        line_cols = line.split(',')
        yield line_cols[1], float(line_cols[2])

    def combiner_count_product(self, product, counts):
        # consolidates all key-value pairs of mapper function (performed at mapper nodes)
        yield product, sum(counts)

    def reducer_count_product(self, product, counts):
        # final consolidation of key-value pairs at reducer nodes
        yield None, '${:,.2f}'.format(sum(counts), product)

    def reducer_find_max(self, _, product_count_pairs):
        yield max(product_count_pairs)

    def steps(self):
        return [
            MRStep(mapper=self.mapper_get_product,
                   combiner=self.combiner_count_product,
                   reducer=self.reducer_count_product),
            MRStep(reducer=self.reducer_find_max)
        ]       

if __name__ == '__main__':
    productRevenue.run()

我收到错误：ValueError: too many values to unpack (expected 2) 但我不知道为什么。 有什么建议？ 我正在运行的 cmd 行代码是：python test.py data.csv .CSV 可以在这里下载https://users.cs.fiu.edu/~prabakar/database/4722sp19/abarr054-3495916/

Answer 1

在 reducer_find_max() 函数中，您在 yield 中传递一个参数。 yield 函数基本上输出一个键和一个值。

请传递两个参数作为“yield max(product_count_pairs), None”