python list conprehension error: global name is not defined

Question

I came across a weird issue. The error message is : global name 'id2tag' is not defined. I have read this post Accessing class variables from a list comprehension in the class definition . But obviously id2tag is not a class variable. The code is listed below.

class evaluater:
    def evaluate_topk(self, ground_truth_dict, predicted_dict, setting_name, setting, data,
                      eval_root_dir = './', file_name = 'result',k = 5,output_entry = 100 ):
        #this part of code is not relevant
        recall = {}
        for key, ground_truth in ground_truth_dict.items():
            recall[key] = recall_at_k(ground_truth, predicted_dict[key])

        mean_recall = np.mean([value for key,value in recall.items()])

        filepath = eval_root_dir + file_name
        if not os.path.exists(os.path.dirname(filepath)):
            os.makedirs(os.path.dirname(filepath))        

        #HERE IS WHERE id2tag DEFINED
        id2tag = {row[0]:row[1] for row in data[config.TYPE_TAG] }
        with open( filepath , 'a' ) as file:
            for itemid in sorted(ground_truth_dict.keys())[1:100]:
                file.write(str(itemid) + '\n')

                file.write('gnd:'+str([id2tag[id] for id in ground_truth_dict[itemid]] ) + '\n')
                file.write('prt' + str([ id2tag[id] for id in predicted_dict[itemid]]) + '\n' )
                #if i use the below code instead, then would be fine
                #gnd_tags = []
                #prt_tags = []
                #for id in ground_truth_dict[itemid]:
                #    gnd_tags.append(id2tag[id])
                #    
                #for id in predicted_dict[itemid]:
                #    prt_tags.append(id2tag[id])
                #    
                #file.write('gnd:'+str( gnd_tags ) + '\n')
                #file.write('prt' + str(prt_tags) + '\n' )

        return mean_recall

Answer 1

Syntax error in your complex embedded code. I think your weird issue will disappear, if you split your one line code to three:

# Change this line 
id2tag = {row[0]:row[1] for row in data[config.TYPE_TAG]}

# Change to this
id2tag = {}
for row in data[config.TYPE_TAG]:
    id2tag[row[0]] = row[1]

BTW: I'd suggest not use embedded for x in list_var in your way, which is not very easy to read and understand for other code readers.

Answer 2

This is a weird issue. I encountered it when debugging inside namespace of a class function in py.test in pdb. Outside of pdb there was no error.

(Pdb) lines=["oneline",2,3,4,5]
(Pdb) ip_dict = dict( ( lines[i], lines[i+1] ) for i in range(0,len(lines),2) )
*** NameError: global name 'lines' is not defined
(Pdb) for i in range(0,len(lines),2): print "%d=%s" % (i,lines[i])
0=oneline
2=3
4=5

(Pdb) self
<Docker-sanity_regression.SANITY testMethod=test_SANITY_002_####_container>

To simplify the problem here is an example.

lines=["oneline",2,3,4,5]
ip_dict = dict( ( lines[i], lines[i+1] ) for i in range(0,len(lines),2) )
print ip_dict

This works fine for python 2.4. And python 2.7. Embedding the dict set in a class func:

class Test:
    def __init__(self):
        print "init"
        self.ip_dict = {}

    def setDict(self):
        lines=["oneline",2,3,4,5]
        self.ip_dict = dict( ( lines[i], lines[i+1] ) for i in range(0,len(lines),2) )
        print self.ip_dict

foo = Test()
foo.setDict()

This also works fine for python 2.4. And python 2.7.

So I am thinking there is a funny namespace problem in certain versions of python and pdb for this type of comprehension. EDIT: Yes. Known issue for pdb. Thanks to https://stackoverflow.com/users/396967/kynan for explaining. https://bugs.python.org/issue21161 Title: list comprehensions don't see local variables in pdb in python3