如何让列表理解更快？

Question

Is there anything in general i could use to speed up nested list comprehensions ( sometimes conditional)?

I was thinking about numpy or transposing (but how?), but there might be anything ive overseen.

Examples:

nextInt = 1
winPoints  = [[max(upSlice[:i])  for i in range(nextInt,len(upSlice)+1)] for upSlice in ppValues]```

or

winningRatio =[ [1 if ratioUp >ratioDown  else 0 if (ratioDown>ratioUp) else
                1 if (pointsUp>pointsDown) else 0 if (pointsDown>pointsUp) else 1   
                for ratioUp,ratioDown,pointsUp,pointsDown                  in  zip(ratioUpSlice,ratioDownSlice,pointsUpSlice,pointsDownSlice)] 
                for ratioUpSlice,ratioDownSlice,pointsUpSlice,pointsDownSlice in  zip(ratios_Up,ratios_Down, pointsUpSlices,pointsDownSlices)]

(all slices in the nested list do NOT have the same lenght)

Answer 1

I'm assuming ppValues is a 2-dimensional list of numbers?

Generally, to speed up algorithms you want to shave off any redundant calculation. For example, in your first piece of code, the max(upSlice[:i]) part is executed (n*m) times and is executing approximately ((n^2)/2 * m) calculations where (n) is the length of the inner list and m is the length of the outer list. And the execution is calculating the max of the same slice (except wIth 1 additional element) which is a lot of unnecessary comparisons.

To speed it up, we can use dynamic programming, where we build a slice_max array such that the element at index i is the max(upSlice[:i]) . And we can do this efficiently by utilizing past computations since the element at index i is equal to the max of two elements, max(slice_max[i-1], upSlice[i]) which is only a single computation.

Here's a simple implementation of the dynamic programming version:

def get_slice_max(arr, start):
    result = [max(arr[:start])]
    for i in range(start, len(arr)):
        result.append(max(result[-1], arr[i]))
    return result
winPoints  = [get_slice_max(upSlice, nextInt) for upSlice in ppValues]

Here's a comparison:

Generating Data

# generate fake data
np.random.seed(42)
# the variable length of inner lists: [7270,  860, 5390, ..., 1389, 4276, 1249]
inner_sizes = np.random.randint(low=1, high=1000, size=10000)
ppValues = [np.random.randint(1000, size=i) for i in inner_sizes]
# ppValues contains 10000 lists, each having 1 to 1000 elements, each element is a number between 1 to 10000

Your version

nextInt=1
winPoints  = [[max(upSlice[:i])  for i in range(nextInt,len(upSlice)+1)] for upSlice in ppValues]

------------------
Wall time: 2min 36s

Simple Dynamic Programming Version

def get_slice_max(arr, start):
    result = [max(arr[:start])]
    for i in range(start, len(arr)):
        result.append(max(result[-1], arr[i]))
    return result
winPoints  = [get_slice_max(upSlice, nextInt) for upSlice in ppValues]

-----------------
Wall time: 1.79 s

You can see a major improvement from 2.5 minutes to <2 seconds.

As for the second example, you did not provide enough context to try to tackle that problem.