Python 3有效地迭代列表的字典

Question

I have working code that classifies data based on rules inside a dictionary of list. I want to know if it is possible to make the code more efficient by getting rid of the nested for loops using list/dictionary comprehensions or .values().

import pandas as pd


df=pd.DataFrame({'Animals': [ 'Python', 'Anaconda', 'Viper', 'Cardinal',
                 'Trout', 'Robin', 'Bass', 'Salmon', 'Turkey', 'Chicken'],
                 'Noise': ['Hiss','SSS','Hisss','Chirp','Splash','Chirp', 
                 'Gulp','Splash','Gobble','Cluck'],
                 })


snakenoise =['Hiss','SSS','Hisss', 'Wissss', 'tseee']
birdnoise =['Chirp', 'squeak', 'Cluck', 'Gobble']
fishnoise =['Splash', 'Gulp', 'Swim']


AnimalDex = {'Snake':['0', 'slither',snakenoise],
              'Bird':['2','fly', birdnoise],
              'Fish':['0','swim',fishnoise],
              }

df['movement'] = ''

for key, value in AnimalDex.items():
    for i in range(len(AnimalDex[key][2])):
        df.loc[df.Noise.str.contains(AnimalDex[key][2][i]),'movement'] = AnimalDex[key][1]

print (df)

Here is the output

    Animals   Noise movement
0    Python    Hiss  slither
1  Anaconda     SSS  slither
2     Viper   Hisss  slither
3  Cardinal   Chirp      fly
4     Trout  Splash     swim
5     Robin   Chirp      fly
6      Bass    Gulp     swim
7    Salmon  Splash     swim
8    Turkey  Gobble      fly
9   Chicken   Cluck      fly

Answer 1

If you just use the values instead of the keys and indexes, you can really simplify your loop.

for animal in AnimalDex.values():
    for value in animal[2]:
        df.loc[df.Noise.str.contains(value),'movement'] = animal[1]

Answer 2

Efficiency does not come from rewriting loops as comprehensions, since comprehensions mainly provide a nicer syntax for loops. Rather, it's the efficiency of the data structure lookups that is important. The problem is that df.Noise.str.contains(AnimalDex[key][2][i]) performs brute-force matching.

If your goal is to merge the movements defined in AnimalDex into the df , joining according to the noise, then it pays to build a dictionary that maps noises to movements:

noise_to_movement = {}
for order in AnimalDex.values():
    for noise in order[2]:
        noise_to_movement[noise] = order[1]

For comparison, here is another way to construct noise_to_movement , using incomprehensible comprehensions:

import itertools

noise_to_movement = dict(itertools.chain(*[list(
    itertools.product(order[2], [order[1]])) for order in AnimalDex.values()
]))

Either way, once the dictionary is built, setting the 'movement' column becomes a trivial lookup:

df['movement'] = list(noise_to_movement[n] for n in df.Noise)

Answer 3

To really improve performance you shouldn't be iterating through a dictionary at all. Instead make a pandas.DataFrame out of that data, and join the two DataFrames.

import pandas as pd

df = pd.DataFrame({'Animals': [ 'Python', 'Anaconda', 'Viper',   'Cardinal',
                   'Trout', 'Robin', 'Bass', 'Salmon', 'Turkey', 'Chicken'],
                   'Noise': ['Hiss','SSS','Hisss','Chirp','Splash','Chirp', 
                   'Gulp','Splash','Gobble','Cluck']})

snakenoise =['Hiss','SSS','Hisss', 'Wissss', 'tseee']
birdnoise =['Chirp', 'squeak', 'Cluck', 'Gobble']
fishnoise =['Splash', 'Gulp', 'Swim']

noises = [(snakenoise, 'Snake', '0', 'slither'),
          (birdnoise, 'Bird', '2', 'fly'),
          (fishnoise, 'Fish', '0', 'swim')]

animal_dex = {'Animal Type': [],
              'Whatever': [],
              'Movement': [],
              'Noise': []}

for noise in noises:
    animal_dex['Noise'] += noise[0]
    animal_dex['Animal Type'] += map(lambda x: noise[1], noise[0])
    animal_dex['Whatever'] += map(lambda x: noise[2], noise[0])
    animal_dex['Movement'] += map(lambda x: noise[3], noise[0])

df1 = pd.DataFrame(animal_dex)

df = df.merge(df1, on='Noise')
df
    Animals   Noise Animal Type Movement Whatever
0    Python    Hiss       Snake  slither        0
1  Anaconda     SSS       Snake  slither        0
2     Viper   Hisss       Snake  slither        0
3  Cardinal   Chirp        Bird      fly        2
4     Robin   Chirp        Bird      fly        2
5     Trout  Splash        Fish     swim        0
6    Salmon  Splash        Fish     swim        0
7      Bass    Gulp        Fish     swim        0
8    Turkey  Gobble        Bird      fly        2
9   Chicken   Cluck        Bird      fly        2