用python中元组列表中元组的第一个元素索引元素的最快方法

Question

list_ = [(1, 'a'), (2, 'b'), (3, 'c')]
item1 = 1
item2 = 'c'
#hypothetical:
assert list_.index_by_first_value(item1) == 0
assert list_.index_by_second_value(item2) == 2

What would be the fastest way to emulate the index_by_first/second_value method in python?

If you don't understand what's going on; if you have a list of tuples (as is contained in list_ ), how would you go about finding the index of a tuple with the first/second value of the tuple being the element you want to index?

---

My best guess would be this:

[i[0] for i in list_].index(item1)
[i[1] for i in list_].index(item2)

But I'm interested in seeing what you guys will come up with. Any ideas?

Answer 1

At first, I thought along the same lines as Nick T . Your method is fine if the number of tuples (N) is short. But of course a linear search is O(N). As the number of tuples increases, the time increases directly with it. You can get O(1) lookup time with a dict mapping the zeroth element of each tuple to its index:

{el[0]:idx for idx,el in enumerate(list_)}

But the cost of converting the list to a dict may be too high! Here are my results:

>>> from timeit import timeit as t
>>> t('[i[0] for i in list_].index(1)', "import random;list_=[(i,'a') for i in range(10)]; random.shuffle(list_)")
1.557116985321045
>>> t('[i[0] for i in list_].index(1)', "import random;list_=[(i,'a') for i in range(100)]; random.shuffle(list_)")
7.415766954421997
>>> t('{el[0]:idx for idx,el in enumerate(list_)}[1]', "import random;list_=[(i,'a') for i in range(10)]; random.shuffle(list_)")
2.1753010749816895
>>> t('{el[0]:idx for idx,el in enumerate(list_)}[1]', "import random;list_=[(i,'a') for i in range(100)]; random.shuffle(list_)")
15.062835216522217

So the list-to-dict conversion is killing any benefit we get from having the O(1) lookups. But just to prove that the dict is really fast if we can avoid doing the conversion more than once:

>>> t('dict_[1]', "import random;list_=[(i,'a') for i in range(10)];random.shuffle(list_);dict_={el[0]:idx for idx,el in enumerate(list_)}")
0.050583839416503906
>>> t('dict_[1]', "import random;list_=[(i,'a') for i in range(100)];random.shuffle(list_);dict_={el[0]:idx for idx,el in enumerate(list_)}")
0.05001211166381836
>>> t('dict_[1]', "import random;list_=[(i,'a') for i in range(1000)];random.shuffle(list_);dict_={el[0]:idx for idx,el in enumerate(list_)}")
0.050894975662231445

Answer 2

Searching a list is O(n). Convert it to a dictionary, then lookups take O(1).

>>> list_ = [(1, 'a'), (2, 'b'), (3, 'c')]
>>> dict(list_)
{1: 'a', 2: 'b', 3: 'c'}
>>> dict((k, v) for v, k in list_)
{'a': 1, 'c': 3, 'b': 2}

If you want the original index you could enumerate it:

>>> dict((kv[0], (i, kv[1])) for i, kv in enumerate(list_))
{1: (0, 'a'), 2: (1, 'b'), 3: (2, 'c')}

>> dict((kv[1], (i, kv[0])) for i, kv in enumerate(list_))
{'a': (0, 1), 'c': (2, 3), 'b': (1, 2)}

Answer 3

EDIT: Just kidding. As the lists grow longer it looks like the manual for loop takes less time. Updated to generate random lists via kojiro's method:

Just some timing tests for your information while maintaining lists. The good thing about preserving list form versus a dictionary is that it's expansible to include tuples of any length.

import timeit
from operator import itemgetter
import random

list_= [('a', i) for i in range(10)]
random.shuffle(list_)

def a():
    return [i[1] for i in list_].index(1)

def b():
    return zip(*list_)[1].index(1)

def c():
    return map(itemgetter(1), list_).index(1)

def d():
     for index, value in enumerate(list_):
         if 1 == value[1]:
             return index

With timeit :

C:\Users\Jesse\Desktop>python -m timeit -s "import test" "test.a()"
1000000 loops, best of 3: 1.21 usec per loop

C:\Users\Jesse\Desktop>python -m timeit -s "import test" "test.b()"
1000000 loops, best of 3: 1.2 usec per loop

C:\Users\Jesse\Desktop>python -m timeit -s "import test" "test.c()"
1000000 loops, best of 3: 1.45 usec per loop

C:\Users\Jesse\Desktop>python -m timeit -s "import test" "test.d()"
1000000 loops, best of 3: 0.922 usec per loop

Answer 4

What is fastest? It depends on how many times you need to use it, and if you are able to create an index dictionary from the very beginning.

As the others have mentioned, dictionary is much faster once you have it, but it is costly to transform the list into a dictionary. I'm going to show what I get on my computer so that I have numbers to compare to. Here's what I got:

>>> import timeit
>>> timeit.timeit('mydict = {val[0]:(ind, val[1]) for ind, val in enumerate(mylist)}', 'mylist = [(i, "a") for i in range(1000)]')
200.36049539601527

Surprisingly, this is significantly slower than it was even to create the list in the first place:

>>> timeit.timeit('mylist = [(i, "a") for i in range(1000)]')
70.15259253453814

So how does this compare to creating a dictionary in the first place?

>>> timeit.timeit('mydict = {i:("a", i) for i in range(1000)}')
90.78464277950229

Obviously, this is not always possible because you are not always the one creating the list, but I wanted to include this for comparisons.

Summary of initializations:

• Creating a list - 70.15
• Creating a dictionary - 90.78
• Indexing an existing list - 70.15 + 200.36 = 270.51

So now, supposing you have a list or dictionary already set up, how long does it take?

>>> timeit.timeit('[i[0] for i in mylist].index(random.randint(0,999))', 'import random; mylist = [(i, "a") for i in range(1000)]')
68.15473008213394

However, this creates a new temporary list each time, so let's look at the breakdown

>>> timeit.timeit('indexed = [i[0] for i in mylist]', 'import random; mylist = [(i, "a") for i in range(1000)];')
55.86422327528999
>>> timeit.timeit('indexed.index(random.randint(0,999))', 'import random; mylist = [(i, "a") for i in range(1000)]; indexed = [i[0] for i in mylist]')
12.302146224677017

55.86 + 12.30 = 68.16, which is consistent with the 68.15 the previous result gave us. Now the dictionary:

>>> timeit.timeit('mydict[random.randint(0,999)]', 'import random; mylist = [(i, "a") for i in range(1000)]; mydict = {val[0]:(ind, val[1]) for ind, val in enumerate(mylist)}')
1.5201382921450204

Of course, in each of these cases I'm using random.randint so let's time that to factor it out:

>>> timeit.timeit('random.randint(0,999)', 'import random')
1.4206546251180043

So now a summary of using the index:

• Using a list - (68.16-1.42) = 66.74 first time, (12.30-1.42) = 10.88 after that
• Using a dictionary - (1.52-1.42) = 0.10 each time

Now let's figure out how many accesses it takes for the dictionary to become more useful. First, a formula for time as a function of number of accesses:

• List - 55.86 + 10.88x
• Dictionary - 200.36 + 0.10x
• Initial dictionary - 20.63 + 0.10x

Based on these formulas, a dictionary becomes faster if you need to access it at least 14 times. If you can create a dictionary from the get-go instead of a list, then the extra overhead to create a dictionary instead of a list is more than offset by the overhead to create a list of just the first values in the tuples.

So which is fastest? It depends on how many times you need to use it, and if you are able to create an index dictionary from the very beginning.

Note: I'm using Python 2.7.5. Timings in Python 3.x could be very different, and also will probably be different on different machines. I'd be curious to see what someone else would come up with on their machine.

All times are in seconds, but timed for one million runs. So individual runs are about the same number in microseconds.

Answer 5

@Nick T

I think some time is wasted enumerating the list and then converting it to a dictionary, so even if it is an O(1) lookup for a dict, creating the dict in the first place is too costly to consider it a viable option for large lists.

This is the test I used to determine it:

import time
l = [(i, chr(i)) for i in range(1000000)]
def test1():
    t1 = time.time()
    ([i[0] for i in l].index(10872))
    t2 = time.time()
    return t2 - t1

def test2():
    t1 = time.time()
    (dict((kv[0], (i, kv[1])) for i, kv in enumerate(l))[10872][0])
    t2 = time.time()
    return t2 - t1

def test3():
    sum1 = []
    sum2 = []
    for i in range(1000):
        sum1.append(test1())
        sum2.append(test2())
    print(sum(sum1)/1000)
    print(sum(sum2)/1000)

test3()

EDIT: Haha Kojiro, you beat me to it!