循环工作时间过长

Question

I have two list of dicts: prices_distincts , prices .

They connect through hash_brand_artnum , both of them sorted by hash_brand_artnum I do not understand why loop works for so long:

1. If length of prices_distincts is 100,000 it works for 30 min

2. But If length of prices_distincts is 10,000 it works for 10 sec .

Code:

 prices_distincts = [{'hash_brand_artnum':1202},...,..]
 prices = [{'hash_brand_artnum':1202,'price':12.077},...,...]

 for prices_distinct in prices_distincts:
    for price in list(prices):            
        if prices_distinct['hash_brand_artnum'] == price['hash_brand_artnum']:

            print price['hash_brand_artnum']
            #print prices
            del prices[0]
        else:
            continue

I need to look for items with same prices. Relation beatween prices_distincts and prices one to many. And group price with equal price['hash_brand_artnum']

Answer 1

it's working so long because your algorithm is O(N^2) and 100000 ^ 2 = 10000000000 and 10000 ^ 2 = 100000000. So factor between two number is 100, and factor between 30 min and 10 sec ~100.

EDIT: It's hard to say by your code and such a small amount of data, and I don't know what your task is, but I think that your dictionaries is not very useful. May be try this:

>>> prices_distincts = [{'hash_brand_artnum':1202}, {'hash_brand_artnum':14}]
>>> prices = [{'hash_brand_artnum':1202, 'price':12.077}, {'hash_brand_artnum':14, 'price':15}]
# turning first list of dicts into simple list of numbers
>>> dist = [x['hash_brand_artnum'] for x in prices_distincts]
# turning second list of dicts into dict where number is a key and price is a value
>>> pr = {x['hash_brand_artnum']:x["price"] for x in prices}

not you can iterate throuth your number and get prices:

>>> for d in dist:
...     print d, pr[d]

Answer 2

As @RomanPekar mentioned, your algorithm is running slow because its complexity is O(n^2) . To fix it, you should write it as an O(n) algorithm:

import itertools as it

for price, prices_distinct in it.izip(prices, prices_distincts):
    if prices_distinct['hash_brand_artnum'] == price['hash_brand_artnum']:
        # do stuff

Answer 3

If prices grows more or less with prices_distincts, then if you multiply the size of prices_distincts by 10, your original 10 seconds will be multiply by 10 then again by 10 (second for loop), and then by ~2 because of the "list(prices)" (that, by the way, should definitively be done out of the loop):

10sec*10*10*2 = 2000sec = 33min

This conversion is usually expensive.

 prices_distincts = [{'hash_brand_artnum':1202},...,..]
 prices = [{'hash_brand_artnum':1202,'price':12.077},...,...]
 list_prices = list(prices)

 for prices_distinct in prices_distincts:
    for price in list_prices:            
        if prices_distinct['hash_brand_artnum'] == price['hash_brand_artnum']:

            print price['hash_brand_artnum']
            #print prices
            del prices[0]
        else:
            continue