Numba: double free or corruption (!prev) Aborted (core dumped)

Question

I'm trying to speed up the following function using numba.

import numpy as np
from numba import jit, prange

@jit(nopython=True, parallel=True)
def find_reg_numba(states):
    reg = []    
    states_sum = np.sum(states, axis=1)
    for i in prange(states.shape[0]):
        if states_sum[i] > 0 and states_sum[i] < 5:
            reg.append(states[i])
    return reg

The states is generated using the following function

def generate_states(size):
    # size is a natural number
    states = np.array(list(map(list, itertools.product([0., 1.], repeat = size))))
    return states

When I try to use the find_reg function, I get the following error trace.

double free or corruption (!prev)
Aborted (core dumped)

My numba version is 0.48.0 .

How to solve this issue?

Answer 1

Not sure why your code produces an error. Related error posts are:

• numpy double free or corruption (!prev) segmentation fault

• Error in `python': double free or corruption (!prev)

• Python double free error for huge datasets

However, these proved unhelpful.

Here's an alternative Numba version of find_reg_numba which:

1. Runs without errors
2. Produces the same result as the original code without Numba (ie original produces errors only with Numba).

Code Refactoring

import numpy as np
from numba import jit
import itertools

@jit(nopython=True, parallel=True)
def find_reg_numba(states):
    states_sum = np.sum(states, axis=1)

    # Find indexes satisfying condition using np.where as described https://www.geeksforgeeks.org/numpy-where-in-python/
    indexes = np.where((states_sum > 0) & (states_sum < 5))
    return states[indexes]

def generate_states(size):
    # size is a natural number
    states = np.array(list(map(list, itertools.product([0., 1.], repeat = size))))
    return states

Tests

for size in range(10):
  s = generate_states(size)
  r  = find_reg_numba(s)
  print(f'Size: {size}\n Result: \n{r}')

Results

 Size: 0
 Result:
[]
Size: 1
 Result:
[[1.]]
Size: 2
 Result:
[[0. 1.]
 [1. 0.]
 [1. 1.]]
Size: 3
 Result:
[[0. 0. 1.]
 [0. 1. 0.]
 [0. 1. 1.]
 [1. 0. 0.]
 [1. 0. 1.]
 [1. 1. 0.]
 [1. 1. 1.]]
Size: 4
 Result:
[[0. 0. 0. 1.]
 [0. 0. 1. 0.]
 [0. 0. 1. 1.]
 [0. 1. 0. 0.]
 [0. 1. 0. 1.]
 [0. 1. 1. 0.]
 [0. 1. 1. 1.]
 [1. 0. 0. 0.]
 [1. 0. 0. 1.]
 [1. 0. 1. 0.]
 [1. 0. 1. 1.]
 [1. 1. 0. 0.]
 [1. 1. 0. 1.]
 [1. 1. 1. 0.]
 [1. 1. 1. 1.]]
Size: 5
 Result:
[[0. 0. 0. 0. 1.]
 [0. 0. 0. 1. 0.]
 [0. 0. 0. 1. 1.]
 [0. 0. 1. 0. 0.]
 [0. 0. 1. 0. 1.]
 [0. 0. 1. 1. 0.]
 [0. 0. 1. 1. 1.]
 [0. 1. 0. 0. 0.]
 [0. 1. 0. 0. 1.]
 [0. 1. 0. 1. 0.]
 [0. 1. 0. 1. 1.]
 [0. 1. 1. 0. 0.]
 [0. 1. 1. 0. 1.]
 [0. 1. 1. 1. 0.]
 [0. 1. 1. 1. 1.]
 [1. 0. 0. 0. 0.]
 [1. 0. 0. 0. 1.]
 [1. 0. 0. 1. 0.]
 [1. 0. 0. 1. 1.]
 [1. 0. 1. 0. 0.]
 [1. 0. 1. 0. 1.]
 [1. 0. 1. 1. 0.]
 [1. 0. 1. 1. 1.]
 [1. 1. 0. 0. 0.]
 [1. 1. 0. 0. 1.]
 [1. 1. 0. 1. 0.]
 [1. 1. 0. 1. 1.]
 [1. 1. 1. 0. 0.]
 [1. 1. 1. 0. 1.]
 [1. 1. 1. 1. 0.]]
Size: 6
 Result:
[[0. 0. 0. 0. 0. 1.]
 [0. 0. 0. 0. 1. 0.]
 [0. 0. 0. 0. 1. 1.]
 [0. 0. 0. 1. 0. 0.]
 [0. 0. 0. 1. 0. 1.]
 [0. 0. 0. 1. 1. 0.]
 [0. 0. 0. 1. 1. 1.]
 [0. 0. 1. 0. 0. 0.]
 [0. 0. 1. 0. 0. 1.]
 [0. 0. 1. 0. 1. 0.]
 [0. 0. 1. 0. 1. 1.]
 [0. 0. 1. 1. 0. 0.]
 [0. 0. 1. 1. 0. 1.]
 [0. 0. 1. 1. 1. 0.]
 [0. 0. 1. 1. 1. 1.]
 [0. 1. 0. 0. 0. 0.]
 [0. 1. 0. 0. 0. 1.]
 [0. 1. 0. 0. 1. 0.]
 [0. 1. 0. 0. 1. 1.]
 [0. 1. 0. 1. 0. 0.]
 [0. 1. 0. 1. 0. 1.]
 [0. 1. 0. 1. 1. 0.]
 [0. 1. 0. 1. 1. 1.]
 [0. 1. 1. 0. 0. 0.]
 [0. 1. 1. 0. 0. 1.]
 [0. 1. 1. 0. 1. 0.]
 [0. 1. 1. 0. 1. 1.]
 [0. 1. 1. 1. 0. 0.]
 [0. 1. 1. 1. 0. 1.]
 [0. 1. 1. 1. 1. 0.]
 [1. 0. 0. 0. 0. 0.]
 [1. 0. 0. 0. 0. 1.]
 [1. 0. 0. 0. 1. 0.]
 [1. 0. 0. 0. 1. 1.]
 [1. 0. 0. 1. 0. 0.]
 [1. 0. 0. 1. 0. 1.]
 [1. 0. 0. 1. 1. 0.]
 [1. 0. 0. 1. 1. 1.]
 [1. 0. 1. 0. 0. 0.]
 [1. 0. 1. 0. 0. 1.]
 [1. 0. 1. 0. 1. 0.]
 [1. 0. 1. 0. 1. 1.]
 [1. 0. 1. 1. 0. 0.]
 [1. 0. 1. 1. 0. 1.]
 [1. 0. 1. 1. 1. 0.]
 [1. 1. 0. 0. 0. 0.]
 [1. 1. 0. 0. 0. 1.]
 [1. 1. 0. 0. 1. 0.]
 [1. 1. 0. 0. 1. 1.]
 [1. 1. 0. 1. 0. 0.]
 [1. 1. 0. 1. 0. 1.]
 [1. 1. 0. 1. 1. 0.]
 [1. 1. 1. 0. 0. 0.]
 [1. 1. 1. 0. 0. 1.]
 [1. 1. 1. 0. 1. 0.]
 [1. 1. 1. 1. 0. 0.]]
Size: 7
 Result:
[[0. 0. 0. 0. 0. 0. 1.]
 [0. 0. 0. 0. 0. 1. 0.]
 [0. 0. 0. 0. 0. 1. 1.]
 [0. 0. 0. 0. 1. 0. 0.]
 [0. 0. 0. 0. 1. 0. 1.]
 [0. 0. 0. 0. 1. 1. 0.]
 [0. 0. 0. 0. 1. 1. 1.]
 [0. 0. 0. 1. 0. 0. 0.]
 [0. 0. 0. 1. 0. 0. 1.]
 [0. 0. 0. 1. 0. 1. 0.]
 [0. 0. 0. 1. 0. 1. 1.]
 [0. 0. 0. 1. 1. 0. 0.]
 [0. 0. 0. 1. 1. 0. 1.]
 [0. 0. 0. 1. 1. 1. 0.]
 [0. 0. 0. 1. 1. 1. 1.]
 [0. 0. 1. 0. 0. 0. 0.]
 [0. 0. 1. 0. 0. 0. 1.]
 [0. 0. 1. 0. 0. 1. 0.]
 [0. 0. 1. 0. 0. 1. 1.]
 [0. 0. 1. 0. 1. 0. 0.]
 [0. 0. 1. 0. 1. 0. 1.]
 [0. 0. 1. 0. 1. 1. 0.]
 [0. 0. 1. 0. 1. 1. 1.]
 [0. 0. 1. 1. 0. 0. 0.]
 [0. 0. 1. 1. 0. 0. 1.]
 [0. 0. 1. 1. 0. 1. 0.]
 [0. 0. 1. 1. 0. 1. 1.]
 [0. 0. 1. 1. 1. 0. 0.]
 [0. 0. 1. 1. 1. 0. 1.]
 [0. 0. 1. 1. 1. 1. 0.]
 [0. 1. 0. 0. 0. 0. 0.]
 [0. 1. 0. 0. 0. 0. 1.]
 [0. 1. 0. 0. 0. 1. 0.]
 [0. 1. 0. 0. 0. 1. 1.]
 [0. 1. 0. 0. 1. 0. 0.]
 [0. 1. 0. 0. 1. 0. 1.]
 [0. 1. 0. 0. 1. 1. 0.]
 [0. 1. 0. 0. 1. 1. 1.]
 [0. 1. 0. 1. 0. 0. 0.]
 [0. 1. 0. 1. 0. 0. 1.]
 [0. 1. 0. 1. 0. 1. 0.]
 [0. 1. 0. 1. 0. 1. 1.]
 [0. 1. 0. 1. 1. 0. 0.]
 [0. 1. 0. 1. 1. 0. 1.]
 [0. 1. 0. 1. 1. 1. 0.]
 [0. 1. 1. 0. 0. 0. 0.]
 [0. 1. 1. 0. 0. 0. 1.]
 [0. 1. 1. 0. 0. 1. 0.]
 [0. 1. 1. 0. 0. 1. 1.]
 [0. 1. 1. 0. 1. 0. 0.]
 [0. 1. 1. 0. 1. 0. 1.]
 [0. 1. 1. 0. 1. 1. 0.]
 [0. 1. 1. 1. 0. 0. 0.]
 [0. 1. 1. 1. 0. 0. 1.]
 [0. 1. 1. 1. 0. 1. 0.]
 [0. 1. 1. 1. 1. 0. 0.]
 [1. 0. 0. 0. 0. 0. 0.]
 [1. 0. 0. 0. 0. 0. 1.]
 [1. 0. 0. 0. 0. 1. 0.]
 [1. 0. 0. 0. 0. 1. 1.]
 [1. 0. 0. 0. 1. 0. 0.]
 [1. 0. 0. 0. 1. 0. 1.]
 [1. 0. 0. 0. 1. 1. 0.]
 [1. 0. 0. 0. 1. 1. 1.]
 [1. 0. 0. 1. 0. 0. 0.]
 [1. 0. 0. 1. 0. 0. 1.]
 [1. 0. 0. 1. 0. 1. 0.]
 [1. 0. 0. 1. 0. 1. 1.]
 [1. 0. 0. 1. 1. 0. 0.]
 [1. 0. 0. 1. 1. 0. 1.]
 [1. 0. 0. 1. 1. 1. 0.]
 [1. 0. 1. 0. 0. 0. 0.]
 [1. 0. 1. 0. 0. 0. 1.]
 [1. 0. 1. 0. 0. 1. 0.]
 [1. 0. 1. 0. 0. 1. 1.]
 [1. 0. 1. 0. 1. 0. 0.]
 [1. 0. 1. 0. 1. 0. 1.]
 [1. 0. 1. 0. 1. 1. 0.]
 [1. 0. 1. 1. 0. 0. 0.]
 [1. 0. 1. 1. 0. 0. 1.]
 [1. 0. 1. 1. 0. 1. 0.]
 [1. 0. 1. 1. 1. 0. 0.]
 [1. 1. 0. 0. 0. 0. 0.]
 [1. 1. 0. 0. 0. 0. 1.]
 [1. 1. 0. 0. 0. 1. 0.]
 [1. 1. 0. 0. 0. 1. 1.]
 [1. 1. 0. 0. 1. 0. 0.]
 [1. 1. 0. 0. 1. 0. 1.]
 [1. 1. 0. 0. 1. 1. 0.]
 [1. 1. 0. 1. 0. 0. 0.]
 [1. 1. 0. 1. 0. 0. 1.]
 [1. 1. 0. 1. 0. 1. 0.]
 [1. 1. 0. 1. 1. 0. 0.]
 [1. 1. 1. 0. 0. 0. 0.]
 [1. 1. 1. 0. 0. 0. 1.]
 [1. 1. 1. 0. 0. 1. 0.]
 [1. 1. 1. 0. 1. 0. 0.]
 [1. 1. 1. 1. 0. 0. 0.]]
Size: 8
 Result:
[[0. 0. 0. ... 0. 0. 1.]
 [0. 0. 0. ... 0. 1. 0.]
 [0. 0. 0. ... 0. 1. 1.]
 ...
 [1. 1. 1. ... 1. 0. 0.]
 [1. 1. 1. ... 0. 0. 0.]
 [1. 1. 1. ... 0. 0. 0.]]
Size: 9
 Result:
[[0. 0. 0. ... 0. 0. 1.]
 [0. 0. 0. ... 0. 1. 0.]
 [0. 0. 0. ... 0. 1. 1.]
 ...
 [1. 1. 1. ... 0. 0. 0.]
 [1. 1. 1. ... 0. 0. 0.]
 [1. 1. 1. ... 0. 0. 0.]]