Why does this recursive function rapidly increase memory use?

Question

I wrote the following recursive function in Python to index a list of str values (with possible repetition of the same value more than once in the list). The function takes in a list and returns a dict where each entry of the dictionary is the item list (an str ) and a corresponding int index.

def make_indices(entries):
    def _make_indices(ent, idxs, idx):
        if not ent:
            return idxs
        else:
            _make_indices(ent[1:], idxs, idx) if ent[0] in idxs \
                else _make_indices(ent[1:], dict({ent[0]: idx}, **idxs), idx+1)
    return _make_indices(entries, {}, 0)

I thought this would be an elegant solution, but its memory use rapidly increases with the length of the list. Would someone be able to expain what exactly might be happening that causes this excess memory use?

Answer 1

Slicing a list ent[1:] , will result in a newly allocated shallow copy of the sliced section. Also, as Python doesn't optimize tail end recursion, you're left in a situation where every single slice you make will remain allocated until the outer call terminates.

Try instead calling ent.pop(0) to remove the first element of the list, and then pass the list as ent without slicing. This way, no new allocation is required