I am trying to create a python script and I am stuck with the dictionaries. I have read through some of the other forums but can't seem to get anywhere. I am a very new python programmer so please be gentle.
What I want to do:
1) set up a dictionary like this: {'Name':'userid','jobid:jobid','walltime:walltime,'nodes:nds'}
2) iterate through a list of entries created from and external function call and extract information to populate the dictionary
3) Problem: I cannot figure out how to append entries to the appropriate keys
For example, I want this:
{‘Name’:’jose’,’jobid’:’001,002,003,005’,’walltime:32:00,240:00,04:00,07:00’,’nodes’:32,32,500’}
Notice for one userid, I have multiple jobids, walltimes and nodes.
(len(jobids)==len(walltimes)==len(nodes)
for any one userid but can vary across userids)
I am able to get the script to find the first value for each username, but it never appends. How can I get this to append?
Here is what I have tried
from collections import defaultdict
pdict = defaultdict(list)
start the loop:
# get new values – add these to the dictionary keyed
# on username (create a new entry or append to existing entry)
…
(jobid,userid,jobname, sessid, nds, tsk, walltime,rest)= m.groups()
...
if userid in pdict:
print "DEBUG: %s is currently in the dictionary -- appending entries" %(userid)
pdict[userid][‘jobid’] = pdict[userid][jobid].append(jobid) I
# repeat for nodes, walltime, etc
if not userid in pdict:
print "DEBUG: %s is not in the dictionary creating entry" %(userid)
pdict[userid] = {} # define a dictionary within a dictionary with key off userid
pdict[userid]['jobid'] = jobid
pdict[userid]['jobname'] = jobname
pdict[userid]['nodes'] = nds
pdict[userid]['walltime'] = walltime
I know this is wrong but can't figure out how to get the append to work. I have tried many of the suggestions offered on this site. I need to append (to the dictionary) the most recent values from the loop keyed to userid
Here is an example of the ouput – it does not append multiple entries for each userid but rather takes only the first value for each userid
userid jmreill contains data: {'nodes': '1', 'jobname': 'A10012a_ReMig_Q', 'walltime': '230:0', 'jobid': '1365582'}
userid igorysh contains data: {'nodes': '20', 'jobname': 'emvii_Beam_fwi6', 'walltime': '06:50', 'jobid': '1398100'}
Any suggestions? This should be easy but I can't figure it out!
from collections import defaultdict
pdict = defaultdict(dict)
start the loop:
# get new values – add these to the dictionary keyed
# on username (create a new entry or append to existing entry)
…
(jobid,userid,jobname, sessid, nds, tsk, walltime,rest)= m.groups()
...
if userid in pdict:
print "DEBUG: %s is currently in the dictionary -- appending entries" %(userid)
pdict[userid][jobid].append(jobid)
# repeat for nodes, walltime, etc
if userid not in pdict:
print "DEBUG: %s is not in the dictionary creating entry" %(userid)
pdict[userid]['jobid'] = [jobid]
pdict[userid]['jobname'] = jobname
pdict[userid]['nodes'] = nds
pdict[userid]['walltime'] = walltime
The value corresponding to key 'jobid'
should be a list of strings rather than a string. If you create your dictionary this way, you can append new jobid's to the list simply by:
pdict[userid]['jobid'].append(jobid)
I cant remember the explanation why to use the lambda expression in the following code, but you have to define a defaultdict
of a defaultdict
:
pdict = defaultdict(lambda: defaultdict(list))
pdict[userid][‘jobid’].append('1234')
will work.
The append()
method does not return the list...it is modified in place. Also, you need to initialize your elements as lists (using square brackets):
if userid in pdict:
print "DEBUG: %s is currently in the dictionary -- appending entries" %(userid)
pdict[userid][jobid].append(jobid) ## just call append here
# repeat for nodes, walltime, etc
if not userid in pdict:
print "DEBUG: %s is not in the dictionary creating entry" %(userid)
pdict[userid] = {} # define a dictionary within a dictionary with key off userid
pdict[userid]['jobid'] = [jobid,] ## initialize with lists
pdict[userid]['jobname'] = [jobname,]
pdict[userid]['nodes'] = [nds,]
pdict[userid]['walltime'] = [walltime,]
append doesn't return a value, it modifies the list in place, and you forgot to quote 'jobid' on the right of equals. So you should replace pdict[userid]['jobid'] = pdict[userid][jobid].append(jobid)
with pdict[userid]['jobid'].append(jobid)
. Also take into account comment from @Jasper.
You are looking for a dict of dicts? AutoVivification
is the perfect solution. Implement the perl's autovivification feature in Python.
class AutoVivification(dict):
"""Implementation of perl's autovivification feature."""
def __getitem__(self, item):
try:
return dict.__getitem__(self, item)
except KeyError:
value = self[item] = type(self)()
return value
This makes everything easier. Note that the value of pdict[userid]['jobid']
should be a list [jobid]
instead of a variable jobid
as you have multiple jobid
.
pdict = AutoVivification()
if userid in pdict:
pdict[userid]['jobid'].append(jobid)
else:
pdict[userid]['jobid'] = [jobid] # a list
Refer to What is the best way to implement nested dictionaries in Python? .
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.