缓存动态生成的函数

Question

I have the probability density functions func1 and func2 (including the support of each) of two random variables. Now I need the probability density function of the sum of these both random variables, which I create via:

import numpy as np
import scipy.integrate
[...]
def density_add(func1, func2, support):
   return np.vectorize(lambda xi: scipy.integrate.simps(func1(support) * func2(xi-support), support))

The problem with that is the huge redundancy. Many values have to be calculated more than once. So I tried to cache but problems appeared due to the dynamically generated functions without unique names.

from joblib import Memory
mem = Memory(cachedir="/tmp/joblib", verbose=0)
[...]
def density_add(func1, func2, support):
   return np.vectorize(mem.cache(lambda xi: scipy.integrate.simps(func1(support) * func2(xi-support), support))


/usr/lib/python3/dist-packages/numpy/lib/function_base.py:2232: JobLibCollisionWarning: Cannot detect name collisions for function '<lambda> [...]
/usr/lib/python3/dist-packages/numpy/lib/function_base.py:2232: JobLibCollisionWarning: Possible name collisions between functions '<lambda>' [...]

What is a better approach to cache such dynamically generated functions?

Answer 1

Could you use functools.lru_cache ? https://docs.python.org/3/library/functools.html#functools.lru_cache . It would be all in memory, so you would lose values between restarts of your program, but the cache would warm up.

from functools import lru_cache

lru_cache as a decorator

>>> @lru_cache()
>>> def myfunc(x):
>>>     print('sleeping')
>>>     return x + 1
>>> myfunc(1)
sleeping
2
>>> myfunc(1)
2

lru_cache as a function

>>> myfunc2 = lru_cache()(lambda x: myfunc(x) *2)
>>> myfunc2(2)
sleeping
6
>>> myfunc2(2)
6