Python动态字典，每个键计数多值

Question

I am writing a python script that parses 1000's of Rancid files that contain all the configuration information, model, software type, software version, and so forth for a bunch of routers on a network. So far I've gotten it to properly loop through all the files and print me out a nice list router by router of the hostname, software type (IE IOS, IOS XR, JUNOSE, JUNOS etc.. ), and what the software version is (IE 12.3R7, 15.2(2)T1, 12.1.1p0.1, etc..).

The problem is, because of the fact that at any point the networking team could add a new router model, upgrade the software to an unknown version, or whatever, managing it by adding variables for every single router type, software version and so on would require constant maintenance on script which I would prefer not to do, so I made all the variables dynamic. It loops through the files, finds the proper software type, version, and model (because each vendor normally doesn't change how that is presented from version to version). Then assigns the variable as 'model', 'type', and 'version' then prints it as it goes.

I want to be able to add an argparse to my code so that when needed rather than an entire list being printed I can get just aa summary with a count so during each pass of the loop i have it add the values it finds to a multivalue per key dictionary.

Here is how the dictionary is built, and how it gets printed out.

I set the key to the filename which is based off the hostname (so less parsing the raw data for more information)

key = file
mydict.setdefault(key, [])
mydict[key].append(model)
mydict[key].append(type)
mydict[key].append(version)

#here is an example of what the dictionary looks like
print mydict

{'router1': ['model1', 'JUNOS', '12.3R7'], 'router2': ['model1', 'JUNOS', 
'13.3R4'], 'router3': ['model2', 'IOS', '15.2'], 'router4': ['model3', 
'JUNOS', '11.4R1'], 'router5': ['model2', 'IOS', '15.3'], 'router6': 
['model4', 'JUNOSe', '12.1.1p0.1'], 'router7': ['model1', 'JUNOS', 
'12.3R7'], 'router7': ['model1', 'JUNOS', '12.3R7'], 'router8': ['model1', 
'JUNOS', '13.3R4'], 'router9': ['model2', 'IOS', '15.2'], 'router10': 
['model3', 'JUNOS', '11.4R1'], 'router11': ['model2', 'IOS', '15.3'], 
'router12': ['model5', 'JUNOS', '12.3R7']}

What I would like is a way to match all the duplicates where all 3 values are the same, count them, and then print them out in a nicely formated list like this (ignoring the key, because printing it doesn't matter for this exercise)

JUNOS    model1 12.3R7 3
JUNOS    model1 13.3R4 2
JUNOS    model3 11.4R1 2
JUNOS    model5 12.3R7 1
IOS      model2 15.2 2
IOS      model2 15.3 2
JUNOSE   model4 12.1.1p0.1 1

Or even more preferable (but probably much more difficult) would be

JUNOS

model1   12.3R7 2
         13.3R4 2
model3   11.4R1 2
model5   12.3R7 1

JUNOSE

model2   12.1.1p0.1 1

IOS

model2   15.2 2
         15.3 2

Answer 1

Maybe Pandas library can help you to achieve this : https://pandas.pydata.org/index.html
You can convert mydict into a Pandas' DataFrame , then use the groupby() method to get all the groups. Finally use size() to count.

You can do:

import pandas as pd
mydict = {} #your dict here
df = pd.DataFrame.from_dict(mydict,'index')
df.columns=['model','type','version'] #Affect column names to your DF.
print(df.groupby(['type','model','version']).size())

That gives you:

type    model   version   
IOS     model2  15.2          2
                15.3          2
JUNOS   model1  12.3R7        2
                13.3R4        2
        model3  11.4R1        2
        model5  12.3R7        1
JUNOSe  model4  12.1.1p0.1    1

When you are affecting columns names with df.columns=... make sure that it fits to your dict values. You must have as many columns as the length of your values list.

Other example of groupby then size can be found here : Duplicate rows in pandas DF

Edit - Dict structuration
In my opinion it should be better with a more descriptive dict using dict as values instead of a list, like:

{'router1': {'bar': None,
  'foo': None,
  'model': 'model1',
  'type': 'JUNOS',
  'version': '12.3R7'},
 'router2': {'bar': None,
  'foo': None,
  'model': 'model1',
  'type': 'JUNOS',
  'version': '13.3R4'},...}

This way pd.DataFrame.form_dict will automatically affect the columns names. The previous code will be:

import pandas as pd
mydict = {} #your dict of dicts here
df = pd.DataFrame.from_dict(mydict,'index')
print(df.groupby(['type','model','version']).size())