making sure only one list is used from a nested list for each iteration of a for loop

Question

I have two lists of lists, one containing two dates and an id number, and one containing lots of information including an id number and a date. I need to find if that date is in between the two other dates for each id number and if so, write it to a new list. More simply, for each id number, if a date is between the other two dates, write the information to a new list. At the moment, all of the list is being put into the new list, which is wrong (not all of the dates will be between the other two).

mv = [['05/13/2012', '09:54:27', 'U', '#0F', '0006E3DADA', 'T', 'Un\n', 'F3', '13 05 12'], 
      ['05/13/2012', '09:54:28', 'U', '#0F', '0006E3DADA', 'T', 'Un\n', 'F3', '13 05 12'],
      etc]

For mv it is the last date in the list that I am interested in.

datepairs = [['21 05 01', '04 06 01', 'C1'], 
             ['27 07 06', '10 08 06', 'C1'],
             etc]

These are the two dates which the date from mv has to be in between.

visitsbetweendates=[]
for visit in mv:
    for date in datepairs1:
        if date[2]==visit[7]: #if the id number is the same in both lists
            if date[0]<= visit[8] <= date[1]: #if the visit date is between the datepair dates
               if visit not in visitsbetweendates: #if the list is unique
                    visitsbetweendates.append(visit)
                break

What I think might be happening is that date[2], date[0] and date[1] are not all coming from the same list in datepairs each time the loop runs, or something is going wrong with the id numbers. I'm sorry if this isn't particularly clear. Thanks for your help!

EDIT: Here is how I converted the dates to datetime objects, which is done right before the code above.

from datetime import datetime
for v in mv:
    e=datetime.strptime(visit[0],'%m/%d/%Y')
    s=e.strftime('%d %m %y')
    visit.append(s)    
datepairs1=[]
for date in datepairs:
    d=datetime.strptime(str(date[0]),' %d %b %y')
    f=datetime.strptime(str(date[1]),' %d %B %Y')
    e=d.strftime('%d %m %y')
    g=f.strftime('%d %m %y')
    gah=[e,g,date[2].strip(' ')]
    datepairs1.append(gah)

Answer 1

First I would transform both lists to dictionaries, this will make the code a lot more efficient since you will not have to loop the datepairs1 list all over again everytime you start searching for a different key, so this is what I would do:

first convert them to dictionaries:

between_dates = dict([(d[2], (d[0], d[1])) for d in datepairs1])
second_dict = {}
for m in mv:
    key = m[7]
    second_dict.setdefault(key ,[])  # this creates the key with an empty list inside if it doesn't exists yet
    second_dict[key].append((m[0], m[8]))

which will end into a syntax like this:

between_dates = {'C1': ('21 05 01', '04 06 01'), ....}
second_dict = {'C1': [('05/13/2012', '13 05 12'), ('05/13/2012', '13 05 12')]}

Doing this in both lists will make the search a lot faster and easier to debug, Now what @eumiro says is important so you should save the dates as datetimes objects instead, you can do this using datetime.strptime . You can find there the format to transform your string dates to a datetime objects. ie: datetime.strptime('02 06 2011', '%d %m %Y')

so now to compare things up:

visits_between_dates=[]
for key, bd in between_dates.items():
    if second_dict.get(key, None):  # This will ask if the se
        for sl  in second_dict.values():
            if not sl in visits_between_dates and bd[0]<= sl[1] <= bd[1]:
                visitsbetweendates.append(sl)

Maybe you need to append all the information from the mv list but that can be easily added to the dictionary.

Answer 2

Preprocess the dates into the same format that can be compared

for n in range(0,len(datepairs)):
    (d,m,y)=(datepairs[n][1]).split(" ")
    datepairs[n][1]="%d%s%s" % (2000+int(y),m,d)
    (d,m,y)=(datepairs[n][0]).split(" ")
    datepairs[n][0]="%d%s%s" % (2000+int(y),m,d)

for d in range(0,len(mv)):
    (d,m,y)=(mv[n][0]).split("/")
    mv[n][0]="%s%s%s",(y,m,d)

Then your "visit in mv" loop should work