Interaction between functools.partial and functools.reduce

Question

I have a function which I'm trying to make a partial version of, and then apply that partial function in a reduce statement.

The partial function itself works- in that I can call it with a single set of arguments normally and it will return the right answer. However it breaks if applied under reduce

for example;

import functools

test=[1,2,3,4]

def base_function(mult=10,a=0,b=0):
    return(mult*(a+b))

#control function, should act like partial base_function with mult=1
def test_function1(a=0,b=0):
    return(a+b)

#stupid partial function that has destroyed my friday afternoon
test_function2=functools.partial(base_function,mult=1)
print(test_function2.args) #returns ()
print(test_function2.keywords) #returns {'mult':1} 

print(test_function1(a=1,b=2)) #returns 3 as expected
print(test_function2(a=1,b=2)) #returns 3 as expected! horray!

#so the partial function itself works when called normally

print(functools.reduce(test_function1,test,-20)) #returns -10
print(functools.reduce(test_function2,test,-20)) #returns TypeError: base_function() got multiple values for argument 'mult'

as an added bonus if I change the code to make mult a posistional argument and update the partial function as so;

def base_function(mult,a=0,b=0):
    return(mult*(a+b))


def test_function1(a=0,b=0):
    return(a+b)

test_function2=functools.partial(base_function,1)
print(test_function2.args)
print(test_function2.keywords)

The bloody thing works perfectly fine with the reduce statement.

Can anyone explain this to me? The only answers I can find for this type error seem to be applicable where someone calls an something in a function as both a keyword and normal argument- which as far as I'm aware has not happened here- and if that was the case the partial function shouldn't work fine when called normally. The 'solution' of making mult a positional argument is workable, but I'd really rather know what causes this error as apparently I must be misunderstanding something crucial.

Edit: thanks to the posters who answered my question- I now understand why it happened and why reordering the arguments or making them positional solves the problem, So I've marked it as solved.

I do however wonder if there is a 'neat' way of doing this- a partial like operator that doesn't just return a partial object that acts like the desired function if arguments are passed correctly, but literally returns an arbitrary desired function without that arbitrary function needing to be predefined. EG:

import functools

def base_function(mult=10,a=0,b=0):
    return(mult*(a+b))

def test_function1(a=0,b=0):
    return(a+b)

test_function2=partial_like_operator(base_function,mult=1)

print(test_function2==test_function1) #prints true

Answer 1

It is all in the ordering of parameteres. In your test cases where you explicitly bypass the order by using a=1, b=2 versus the reduce() which uses the two first parameters, it does fail.

However if you turn it around, this seems to work:

def base_function(a=0, b=0, mult=10):
    return mult * (a + b)

This seems to do the trick for me, at least!

The following will most likely fail in your case as well:

print(test_function2(1, 2))

This is due to the fact that it tries to set mult twice...

Answer 2

@horoy is right.

Another way to explain that:

Under the hood, the call to reduce which gives an error, ends up with a call like:

test_function2(x, y, mult=1)

where x and y are the local variables of the reduce function.

You can try it and easily see for yourself that this is not a valid call. The first argument passed must correspond to mult because that's the first parameter of the function, but then mult is passed another value, as a kw arg.

In your test calling the partial function which worked, you passed a and b as kwargs, which bypasses this issue.

The conclusion is that when using partial, it is probably best to design it in such a way which binds the last parameters of the functions, not the first.