Recursive function definition in python

Question

So I am trying to implement a version of Hartree-Fock theory for a band system. Basically, it's a matrix convergence problem. I have a matrix H0, from whose eigenvalues I can construct another matrix F. The procedure is then to define H1 = H0 + F and check if the eigenvalues of H1 is close to the ones of H0. If not, I construct a new F from eigenvalues of H1 and define H2 = H0 + F. Then check again and iterate.

The problem is somewhat generic and my exact code seems not really relevant. So I am showing only this:

# define the matrix F
def generate_fock(H):
    def fock(k): #k is a 2D array
        matt = some prefactor*outer(eigenvectors of H(k) with itself) #error1
        return matt
    return fock

k0 = linspace(0,5*kt/2,51)
# H0 is considered defined
H = lambda k: H0(k)
evalold = array([sort(linalg.eigvalsh(H(array([0,2*k-kt/2])))) for k in k0[:]])[the ones I care]
while True:
    fe = generate_fock(H)
    H = lambda k: H0(k)+fe(k) #error2
    evalnew = array([sort(linalg.eigvalsh(H(array([0,2*k-kt/2])))) for k in k0[:]])[the ones I care]
    if allclose(evalnew, evalold): break
    else: evalold = evalnew

I am using inner functions, hoping that python would not find my definitions are recursive (I am not sure if I am using the word correctly). But python knows :( Any suggestions?

Edit1: The error message is highlighting the lines I labeled error1 and error2 and showing the following:

RecursionError: maximum recursion depth exceeded while calling a Python object

I think this comes from my way of defining the functions: In loop n, F(k) depends on H(k) of the previous loop and H(k) in the next step depends on F(k) again. My question is how do I get around this?

Edit2&3: Let me add more details to the code as suggested. This is the shortest thing I can come up with that exactly reproduce my problem.

from numpy import *
from scipy import linalg

# Let's say H0 is any 2m by 2m Hermitian matrix. m = 4 in this case.
# Here are some simplified parameters 
def h(i,k):
    return -6*linalg.norm(k)*array([[0,exp(1j*(angle(k@array([1,1j]))+(-1)**i*0.1/2))],
                                    [exp(-1j*(angle(k@array([1,1j]))+(-1)**i*0.1/2)),0]])
T = array([ones([2,2]),
          [[exp(-1j*2*pi/3),1],[exp(1j*2*pi/3),exp(-1j*2*pi/3)]],
          [[exp(1j*2*pi/3),1],[exp(-1j*2*pi/3),exp(1j*2*pi/3)]]])
g = array([[ 0.27023695,  0.46806412], [-0.27023695,  0.46806412]])
kt = linalg.norm(g[0])
def H0(k):
    "one example"
    matt = linalg.block_diag(h(1,k),h(2,k+g[0]),h(2,k+g[1]),h(2,k+g[0]+g[1]))/2
    for j in range(3): matt[0:2,2*j+2:2*j+4] = T[j]
    return array(matrix(matt).getH())+matt
dim = 4
def bz(x):
    "BZ centered at 0 with (2x)^2 points in it"
    tempList = []
    for i in range(-x,x):
        for j in range(-x,x):
            tempList.append(i*g[0]/2/x+j*g[1]/2/x)
    return tempList
def V(i,G):
    "2D Coulomb interaction"
    if linalg.norm(G)==0: return 0
    if i>=dim: t=1
    else: t=0
    return 2*pi/linalg.norm(G)*exp(0.3*linalg.norm(G)*(-1+(-1)**t)/2)

# define the matrix F for some H
def generate_fock(H):
    def fock(k): #k is a 2D array
        matf = zeros([2*dim,2*dim],dtype=complex128)
        for pt in bz(1): #bz is a list of 2D arrays
            matt = zeros([2*dim,2*dim],dtype=complex128)
            eig_vals1, eig_vecs1 = linalg.eigh(H(pt)) #error1
            idx = eig_vals1.argsort()[::]
            vecs1 = eig_vecs1[:,idx][:dim]
            for vec in vecs1:
                matt = matt + outer(conjugate(vec),vec)
            matt = matt.transpose()/len(bz(1))
            for i in range(2*dim):
                for j in range(2*dim):
                    matf[i,j] = V(j-i,pt-k)*matt[i,j] #V is some prefactor
        return matf
    return fock

k0 = linspace(0,5*kt/2,51)
H = lambda k: H0(k)
evalold = array([sort(linalg.eigvalsh(H(array([0,2*k-kt/2])))) for k in k0[:]])[dim-1:dim+1]
while True:
    fe = generate_fock(H)
    H = lambda k: H0(k)+fe(k) #error2
    evalnew = array([sort(linalg.eigvalsh(H(array([0,2*k-kt/2])))) for k in k0[:]])[dim-1:dim+1]
    if allclose(evalnew, evalold): break
    else: evalold = evalnew

Answer 1

The problem is these lines:

while True:
    fe = generate_fock(H)
    H = lambda k: H0(k)+fe(k) #error2

In each iteration, you are generating a new function referencing the older one rather than the final output of that older function, so it has to keep them all on the stack. This will also be very slow, since you have to back multiply all your matrices every iteration.

What you want to do is keep the output of the old values, probably by making a list from the results of the prior iteration and then applying functions from that list.

Potentially you could even do this with a cache, though it might get huge. Keep a dictionary of inputs to the function and use that. Something like this:

# define the matrix F
def generate_fock(H):
    d = {}
    def fock(k): #k is a 2D array
        if k in d:
            return d[k]
        matt = some prefactor*outer(eigenvectors of H(k) with itself) #error1
        d[k] = matt
        return matt
    return fock

Then it should hopefully only have to reference the last version of the function.

EDIT: Give this a try. As well as caching the result, keep an index into an array of functions instead of a reference. This should prevent a recursion depth overflow.

hList = []

# define the matrix F
def generate_fock(n):
    d = {}
    def fock(k): #k is a 2D array
        if k in d:
            return d[k]
        matt = some prefactor*outer(eigenvectors of hList[n](k) with itself) #error1
        d[k] = matt
        return matt
    return fock

k0 = linspace(0,5*kt/2,51)
# H0 is considered defined
HList.append(lambda k: H0(k))
H = HList[0]
evalold = array([sort(linalg.eigvalsh(H(array([0,2*k-kt/2])))) for k in k0[:]])[the ones I care]
n = 0
while True:
    fe = generate_fock(n)
    n += 1
    hList.append(lambda k: H0(k)+fe(k)) #error2
    H = hList[-1]
    evalnew = array([sort(linalg.eigvalsh(H(array([0,2*k-kt/2])))) for k in k0[:]])[the ones I care]
    if allclose(evalnew, evalold): break
    else: evalold = evalnew