检查另一个字典中是否存在python字典中值列表中的值

Question

I writing a python script to verify report templates. There is a master template, which contains all possible reports with all their possible fields. Then there is a template csv which contains some of those reports. For simplicity lets say then look like this;

Template csv:

OutputName, col1, col2, col3, col4, col5 PersonReport, name, surname, age, dob, id AccountReport, f1, f2, f3, f4, f5 TransactionReport, f1, f2, f3, f4, f5

Master csv:

OutputName, col1, col2, col3, col4, col5 PersonReport, name, surname, street, age, id TransactionReport, f1, f2, f3, f4, f5

So in this example the AccountReport doesn't even exist in the Master, the PersonReport contains a field dob which isn't valid because it isn't in the Master. The only valid report is TransactionReport .

So I idea is to read these csvs in as dictionaries, with the OutputName as the key and the field names as the values.

 import pandas as pd
 masterDf = pd.read_csv('master.csv')
 master = masterDf.set_index('OutputName').T.to_dict('list')

 templateDf = pd.read_csv('template.csv')
 template = templateDf.set_index('OutputName').T.to_dict('list')

The dictionaries looks like;

template = {' PersonReport': [' name', ' surname', ' age', ' dob', ' id'], ' AccountReport': [' f1', ' f2', ' f3', ' f4', ' f5 '], ' TransactionReport': [' f1', ' f2', ' f3', ' f4', ' f5']}

master = {' PersonReport': [' name', ' surname', ' street', ' age', ' id'], ' TransactionReport': [' f1', ' f2', ' f3', ' f4', ' f5']}

Now the I want to first match on the keys, to find out which keys don't exist in the master dict. After that when finding keys that do match, I wish to check if the values in the dictionary are valid, by checking that they exist in the master dict's values.

So I try :

errorCount = 0

for key, value in template.items():
    if key  not in master:
        print("{} is an invalid report".format(key))
        errorCount += 1
    if key in master:
        for fields in template.values():
            for field in fields:
                for cols in master.values():
                    if field not in cols:
                        print("{} is an invalid field in {} report".format(field, key))
                        errorCount += 1

However the output I get is wrong. I get :

name is an invalid field in  PersonReport report
 surname is an invalid field in  PersonReport report
 age is an invalid field in  PersonReport report
 dob is an invalid field in  PersonReport report
 dob is an invalid field in  PersonReport report
 id is an invalid field in  PersonReport report
 f1 is an invalid field in  PersonReport report
 f2 is an invalid field in  PersonReport report
 f3 is an invalid field in  PersonReport report
 f4 is an invalid field in  PersonReport report
 f5  is an invalid field in  PersonReport report
 f5  is an invalid field in  PersonReport report
 f1 is an invalid field in  PersonReport report
 f2 is an invalid field in  PersonReport report
 f3 is an invalid field in  PersonReport report
 f4 is an invalid field in  PersonReport report
 f5 is an invalid field in  PersonReport report
 AccountReport is an invalid report
 name is an invalid field in  TransactionReport report
 surname is an invalid field in  TransactionReport report
 age is an invalid field in  TransactionReport report
 dob is an invalid field in  TransactionReport report
 dob is an invalid field in  TransactionReport report
 id is an invalid field in  TransactionReport report
 f1 is an invalid field in  TransactionReport report
 f2 is an invalid field in  TransactionReport report
 f3 is an invalid field in  TransactionReport report
 f4 is an invalid field in  TransactionReport report
 f5  is an invalid field in  TransactionReport report
 f5  is an invalid field in  TransactionReport report
 f1 is an invalid field in  TransactionReport report
 f2 is an invalid field in  TransactionReport report
 f3 is an invalid field in  TransactionReport report
 f4 is an invalid field in  TransactionReport report
 f5 is an invalid field in  TransactionReport report

My expected output would :

AccountReport is an invalid report
dob is an invalid field in PersonReport report

Any help is appreciated Thanks ps I'm using python 3.6

Answer 1

You can try this:

template = {' PersonReport': [' name', ' surname', ' age', ' dob', ' id'], ' AccountReport': [' f1', ' f2', ' f3', ' f4', ' f5 '], ' TransactionReport': [' f1', ' f2', ' f3', ' f4', ' f5']}
master = {' PersonReport': [' name', ' surname', ' street', ' age', ' id'], ' TransactionReport': [' f1', ' f2', ' f3', ' f4', ' f5']}
invalid_names = ["{} in dict".format(i) if i in master else "{} not in dict".format(i) for i in template]
invalid_values = filter(lambda x:x, [["{} is an invalid value".format(c) for c in b if c not in master[a]] for a, b in template.items() if a in master])
print(invalid_names)
print(invalid_values)

Output:

[' TransactionReport in dict', ' PersonReport in dict', ' AccountReport not in dict']
[[' dob is an invalid value']]

Answer 2

Your code had couple of mistakes which lead to you checking every field. Use of continue and only iterating over the correct key solved this. Here's a fixed code snippet:

for key in template.keys():
    if key not in master:
        print("{} is an invalid report".format(key))
        errorCount += 1
        continue # Continue to next key
    cols = master[key] # Get the correct column names from the current key
    errorFlag = False
    for field in template[key]:
        if field not in cols:
            print("{} is an invalid field in {} report".format(field, key)) 
            errorCount += 1
            errorFlag = True # You can break here if you wish not to keep incrementing the errorCount.

    if not errorFlag: # We did not find any bad column
        print("Success finding valid {} report in master".format(key))

This outputs:

 dob is an invalid field in  PersonReport report
 AccountReport is an invalid report
Success finding valid  TransactionReport report in master

Answer 3

I kept it simple and followed your conventions:

errorCount = 0

for key in template.keys():
    if key not in master:
        print("{} is an invalid report".format(key))
        errorCount += 1
    else:
        if template[key] != master[key]:
            fields = [f for f in template[key] if f not in master[key]]
            for f in fields:
                print("{} is an invalid field in {} report".format(f, key))
                errorCount += 1

From the console:

AccountReport is an invalid report
dob is an invalid field in  PersonReport report