Python-匹配/比较两个数据/元组列表

Question

I have two sets of data, one is from a queryset in django, which will return data as below:

sr_data =  ShowroomConfigData.objects.only('location').filter(is_showroom=True).exclude(location='MajorSite')
for i in sr_data:
    print i.location

London
Glasgow
Edinbrugh
...

The second set of data is from an external Mysql query that returns a list of tuples:

query = """long mysql query..."""
cur.execute(query)
esr_data = cur.fetchall()
for i in esr_data:
    print i[3]

London
Glasgow
Edinburgh
...

esr_data sample:

('John Smith', '0123456789', 'billy', 'London', 'London', datetime.date(2014, 12, 19), '0123456789', 'Bobs Builders', '123 place', 'city', 'add', 'London', 'LDN 103', ', '', '', ' ', '', '', ' ', ' ', ' ', ' ', '', ' ', '')

They're not necessarily in that order either, the orders of both are random I think.

But the external query has the some details I want to import into django on a regular basis.

So I need to loop both lists and import data into django when they match, the only problem is, looping they will likely never match.

Does anyone know of a way I can make this work?

Thanks

Answer 1

So, as I understand it, you have 2 variables which are both iterables. They contain some value and you want to find the items in one when it matches the items in the other.

So, a naive way of doing it is:

for i in esr_data:
    for j in sr_data:
        if i[3] == j.location:
            # Import into django as they match

But this is not very nice as it is O(M * N) where M is num of esr_data and N is num of sr_data

Are you M and N very large ? If not, this could work.

To reduce complexity for large data, first find the common locations in both:

common_locations = set(d.location for d in sr_data) & set(d[3] for d in esr_data)
new_sr_data = filter(lambda d: d.location in common_locations,
                     sr_data) 
new_esr_data = filter(lambda d: d[3] in common_locations,
                      esr_data) 
for i in new_esr_data:
    for j in new_sr_data:
        if i[3] == j.location:
            # Import into django as they match

Which reduces complexity to O(L * L) where L is the number of common elements.

Answer 2

It looks like esr_data returns a tuple of elements, not a list of tuples.

So you just need to compare lists for similar elements. You can sets for this:

result = list(set(sr_data).intersection(esr_data))

Answer 3

Having read through the other answers, I'm beginning to question my own assumptions, and think the actual query is more:

Two lists of tuples, find the rows which are in both, based on specific fields (eg: fields 1, 3 and 5 match in both)

Based on that problem, you can build a set of the "bits that match" from the first list, and then just pull items from the second that match anything in that set, for example, the following filters based on the first and last items in each tuple:

x = [(1,2,3),(2,3,4),(3,4,5),(4,5,6)]
y = [(2,9,4),(4,9,6),(7,8,9),(1,1,1)]

set_x = set((i[0], i[2]) for i in x) # Set of unique keys in x

for item in y:
    if (item[0], item[2]) in set_x:
         pass # Do some importing here

I've included my original answer below, just in case it's of use.

---

Based on the assumption that the problem is:

• There are two iterables of tuples
• "Matching" tuples from each list contain all the same values, but in different orders (ie: the fields are in different orders, but there are the same number of values, and all of those values need to match)
• You want a new list of tuples to import where the values exist in both of the original lists?

One option would be to convert the lists of tuples into sets of (frozen) sets, and then simply pull the intersection - ie: All sets that exist in both sets. For example:

x = [(1,2,3),(2,3,4),(3,4,5),(4,5,6)]
y = [(2,3,4),(4,5,6),(7,8,9),(1,1,1)]
set_x = set(frozenset(i) for i in x)
set_y = set(frozenset(i) for i in y)
matches = set_x.intersection(set_y)