获取嵌套Python字典中特定值的计数

Question

I have a giant nested dictionary (6k records) that I need to sort and count based on two values within my second dict.

item_dict = {
    64762.0: {
        'In Sheet': 'No',
        'Paid': Y,
        'Region': "AMER'",
        'Matrix Position': 'Check'
    },
    130301.0: {
        'Paid': N,
        'Region': "AMER'",
        'Matrix Position': 'Calculate'
    },
    13111.0: {
        'In Sheet': 'Yes',
        'Region': "EMEA'",
        'Matrix Position': 'Check'
    },
    130321.0: {
        'Matrix Position': 'Enhance',
        'In Sheet': 'No',
        'Paid': Y,
        'Region': "JP'"
    }
}

So, I need to get counts between regions and Matrix positions. So, I'd wind up with:

Amer and Calculate: 1
EMEA and Calculate: 0
EMEA and Check= 1
AMER and Check= 1
EMEA and Enhance= 0
JP and Check=0 

Et cetera. The thing is, the full data set has 5 regions with 4 potential matrix positions. Is the best way to do this by using a for loop to search for each potential combination, then adding that to its own list?

AmerCalculate=[]
for row in item_dict:
    if item_dict[row]['Region'] == "AMER'" and item_dict[row]['Matrix Position'] == "Calculate":
        AmerCalculate.append(row)

Then, to get the lengths, do len(AmerCalculate)? Is there a more elegant way of doing this so I don't have to manually type out all 20 combinations?

Answer 1

AmerCalculate={}
Regions = ["AMER", "EMEA", "JP"]
Positions = ["Calculate", "Check"]
for row in item_dict():
    for region in regions:
        for pos in Positions:
            if (item_dict[row]['Region']==region) and (item_dict[row][MatrixPosition] == pos:
    AmerCalculate(str(region)+ ' and ' +str(pos) + ":")+=1

This will return a dictionary with the format as follows: { "region + matrixposition:": total} for example {Amer and Calculate: 1, EMEA and calculate: 1}

Do you need to return the key? or just the totals of each position per region?

Answer 2

Use another dictionary to couple that data set together, from there you can generate the output you're looking for:

def dict_counter(dict_arg):
    d = {'AMER':[],'EMEA':[],'JP':[]}  # Regions as keys.

    for int_key in dict_arg:
        sub_dict = dict_arg[int_key]
        for key, value in sub_dict.items():
            if value in d:
                d[value].append(sub_dict['Matrix Position'])
    return d

Sample Output:

>>> item_dict= {12.0: {'In Sheet': 'No', 'Paid': 'Y', 'Region': "AMER",  'Matrix Position': 'Enhance'},1232.0: {'In Sheet': 'No', 'Paid': 'Y', 'Region': "AMER",  'Matrix Position': 'Check'}, 64762.0: {'In Sheet': 'No', 'Paid': 'Y', 'Region': "AMER",  'Matrix Position': 'Check'}, 130301.0: {'Paid': 'N', 'Region': "AMER",  'Matrix Position': 'Calculate'}, 13111.0: {'In Sheet': 'Yes', 'Region': "EMEA",  'Matrix Position': 'Check'}, 130321.0: {'Matrix Position': 'Enhance','In Sheet': 'No', 'Paid': 'Y', 'Region': "JP"}}
>>> print dict_counter(item_dict)
{'JP': ['Enhance'], 'AMER': ['Check', 'Calculate'], 'EMEA': ['Check']}

We now have the basis to generate the report you're looking for. We can use Counter to get a count of all position instances. Here's an example of how we could go about checking for counts in the list mapped value.

from collections import Counter

d = dict_counter(item_dict)
for k, v in d.items():
    for i, j in Counter(v).items():
        print k,'and',i,'=',j

>>> JP and Enhance = 1
>>> AMER and Enhance = 1
>>> AMER and Check = 2
>>> AMER and Calculate = 1
>>> EMEA and Check = 1

Answer 3

To get all combinations, you can use itertools.product . Then you can store the result in a dictionary:

result = {}
for r, p in itertools.product(regions, positions):
    result[(r,p)] = len( [None for item in item_dict.values() if item['Region'] == r and item['Matrix Position'] == p] )

print(result[("AMER", "Calculate")])

Answer 4

Is it critical for you to use pure Python ? I guess, if you wanted just to do this once, you could do this without care of performance or beauty, either you want to know something new.

What's about pandas library which can solve this problem in fast and elegant way without ugly loops? It allows to group your data in way you want to and manipulate it. For example, this code

data_frame.groupby(['Region', 'Matrix Position'])['Matrix Position'].count()

Will give you what you wanted without doing any loops, not needed subroutines in fast and convenient way

Region  Matrix Position
AMER'   Calculate          1
        Check              1
EMEA'   Check              1
JP'     Enhance            1

It may help you to continue processing/preparation of your data as it has a lot of abilities for data processing and analysis.

One more example: following code will calculate amount of rows with AMER' region and Check matrix position

from pandas import DataFrame

data_frame = DataFrame(item_dict).transpose()
filtered_data = data_frame[(data_frame['Region'] == "AMER'")
                           & (data_frame['Matrix Position'] == 'Check')]
result = len(filtered_data.index)