我正在嘗試編寫一個 python 腳本來使用 Mapreducer 讀取 csv 並且我收到錯誤 ValueError: too many values to unpack (expected 2)

Question

0

我在 MapReduce 中運行以下 Python 代碼：

from mrjob.job import MRJob
from mrjob.step import MRStep


class productRevenue(MRJob):
    #each input lines consists of product, productCategory, price, and paymentMode
    def mapper_get_product(self, _, line):
        # create a key-value pair with key: product and value: price
        line_cols = line.split(',')
        yield line_cols[1], float(line_cols[2])

    def combiner_count_product(self, product, counts):
        # consolidates all key-value pairs of mapper function (performed at mapper nodes)
        yield product, sum(counts)

    def reducer_count_product(self, product, counts):
        # final consolidation of key-value pairs at reducer nodes
        yield None, '${:,.2f}'.format(sum(counts), product)

    def reducer_find_max(self, _, product_count_pairs):
        yield max(product_count_pairs)

    def steps(self):
        return [
            MRStep(mapper=self.mapper_get_product,
                   combiner=self.combiner_count_product,
                   reducer=self.reducer_count_product),
            MRStep(reducer=self.reducer_find_max)
        ]       

if __name__ == '__main__':
    productRevenue.run()

我收到錯誤：ValueError: too many values to unpack (expected 2) 但我不知道為什么。 有什么建議？ 我正在運行的 cmd 行代碼是：python test.py data.csv .CSV 可以在這里下載https://users.cs.fiu.edu/~prabakar/database/4722sp19/abarr054-3495916/

Answer 1

在 reducer_find_max() 函數中，您在 yield 中傳遞一個參數。 yield 函數基本上輸出一個鍵和一個值。

請傳遞兩個參數作為“yield max(product_count_pairs), None”