通过矢量化提高 np.irr function 的性能

Question

Is it possible to improve the performance of the np.irr function such that it can applied to a 2-dimension array of cash flows without using a for-loop--either though vectorizing the np.irr function or through an alternative algorithm?

The irr function in the numpy library calculates the periodically compounded rate of return that gives a net present value of 0 for an array of cash flows. This function can only be applied to a 1-dimensional array:

x = np.array([-100,50,50,50])
r = np.irr(x)

np.irr will not work against a 2-dimensional array of cash flows, such as:

cfs = np.zeros((10000,4))
cfs[:,0] = -100
cfs[:,1:] = 50

where each row represents a series of cash flows, and columns represent time periods. Therefore a slow implementation would be to loop over each row and apply np.irr to individual rows:

out = []
for x in cfs:
    out.append(np.irr(x))

For large arrays, this is an optimization barrier. Looking at the source code of the np.irr function, I believe the main obstacle is vectorizing the np.roots function:

def irr(values):
    res = np.roots(values[::-1])
    mask = (res.imag == 0) & (res.real > 0)
    if res.size == 0:
        return np.nan
    res = res[mask].real
    # NPV(rate) = 0 can have more than one solution so we return
    # only the solution closest to zero.
    rate = 1.0/res - 1
    rate = rate.item(np.argmin(np.abs(rate)))
    return rate

I have found a similar implementation in R: Fast loan rate calculation for a big number of loans , but don't know how to port this into Python. Also, I don't consider np.apply_along_axis or np.vectorize to be solutions to this issue since my main concern is performance, and I understand both are wrappers for a for-loop.

Thanks!

Answer 1

Looking at the source of np.roots ,

import inspect
print(inspect.getsource(np.roots))

We see that it works by finding the eigenvalues of the "companion matrix". It also does some special handling of coefficients that are zero. I really don't understand the mathematical background, but I do know that np.linalg.eigvals can calculate eigenvalues for multiple matrices in a vectorized way.

Merging it with the source of np.irr has resulted in the following "Frankencode":

def irr_vec(cfs):
    # Create companion matrix for every row in `cfs`
    M, N = cfs.shape
    A = np.zeros((M, (N-1)**2))
    A[:,N-1::N] = 1
    A = A.reshape((M,N-1,N-1))
    A[:,0,:] = cfs[:,-2::-1] / -cfs[:,-1:]  # slice [-1:] to keep dims

    # Calculate roots; `eigvals` is a gufunc
    res = np.linalg.eigvals(A)

    # Find the solution that makes the most sense...
    mask = (res.imag == 0) & (res.real > 0)
    res = np.ma.array(res.real, mask=~mask, fill_value=np.nan)
    rate = 1.0/res - 1
    idx = np.argmin(np.abs(rate), axis=1)
    irr = rate[np.arange(M), idx].filled()
    return irr

This does not do handling of zero coefficients and surely fails when any(cfs[:,-1] == 0) . Also some input argument checking wouldn't hurt. And some other problems maybe? But for the supplied example data it achieves what we wanted (at the cost of increased memory use):

In [487]: cfs = np.zeros((10000,4))
     ...: cfs[:,0] = -100
     ...: cfs[:,1:] = 50

In [488]: %timeit [np.irr(x) for x in cfs]
1 loops, best of 3: 2.96 s per loop

In [489]: %timeit irr_vec(cfs)
10 loops, best of 3: 77.8 ms per loop

If you have the special case of loans with a fixed payback amount (like in the question you linked) you may be able do it faster using interpolation...

Answer 2

After I posted this question I worked on this question and came up with a vectorized solution that uses a different algorithm:

def virr(cfs, precision = 0.005, rmin = 0, rmax1 = 0.3, rmax2 = 0.5):
    ''' 
    Vectorized IRR calculator. First calculate a 3D array of the discounted
    cash flows along cash flow series, time period, and discount rate. Sum over time to 
    collapse to a 2D array which gives the NPV along a range of discount rates 
    for each cash flow series. Next, find crossover where NPV is zero--corresponds
    to the lowest real IRR value. For performance, negative IRRs are not calculated
    -- returns "-1", and values are only calculated to an acceptable precision.

    IN:
        cfs - numpy 2d array - rows are cash flow series, cols are time periods
        precision - level of accuracy for the inner IRR band eg 0.005%
        rmin - lower bound of the inner IRR band eg 0%
        rmax1 - upper bound of the inner IRR band eg 30%
        rmax2 - upper bound of the outer IRR band. eg 50% Values in the outer 
                band are calculated to 1% precision, IRRs outside the upper band 
                return the rmax2 value
    OUT:
        r - numpy column array of IRRs for cash flow series
    '''

    if cfs.ndim == 1: 
        cfs = cfs.reshape(1,len(cfs))

    # Range of time periods
    years = np.arange(0,cfs.shape[1])

    # Range of the discount rates
    rates_length1 = int((rmax1 - rmin)/precision) + 1
    rates_length2 = int((rmax2 - rmax1)/0.01)
    rates = np.zeros((rates_length1 + rates_length2,))
    rates[:rates_length1] = np.linspace(0,0.3,rates_length1)
    rates[rates_length1:] = np.linspace(0.31,0.5,rates_length2)

    # Discount rate multiplier rows are years, cols are rates
    drm = (1+rates)**-years[:,np.newaxis]

    # Calculate discounted cfs   
    discounted_cfs = cfs[:,:,np.newaxis] * drm

    # Calculate NPV array by summing over discounted cashflows
    npv = discounted_cfs.sum(axis = 1)

    ## Find where the NPV changes sign, implies an IRR solution
    signs = npv < 0

    # Find the pairwise differences in boolean values when sign crosses over, the
    # pairwise diff will be True
    crossovers = np.diff(signs,1,1)

    # Extract the irr from the first crossover for each row
    irr = np.min(np.ma.masked_equal(rates[1:]* crossovers,0),1)

    # Error handling, negative irrs are returned as "-1", IRRs greater than rmax2 are
    # returned as rmax2
    negative_irrs = cfs.sum(1) < 0
    r = np.where(negative_irrs,-1,irr)
    r = np.where(irr.mask * (negative_irrs == False), 0.5, r)

    return r

Performance:

import numpy as np
cfs = np.zeros((10000,4))
cfs[:,0] = -100
cfs[:,1:] = 50

%timeit [np.irr(x) for x in cfs]
10 loops, best of 3: 1.06 s per loop

%timeit virr(cfs)
10 loops, best of 3: 29.5 ms per loop

Answer 3

pyxirr is super fast, and np.irr is deprecated, so I'd use this now:

https://pypi.org/project/pyxirr/

import pyxirr

cfs = np.zeros((10000,4))
cfs[:,0] = -100
cfs[:,1:] = 50

df = pd.DataFrame(cfs).T
df.apply(pyxirr.irr)