"SystemError: <class 'int'> returned a result with an error set" in Python

Question

I wanted to apply a very simple function using ndimage.generic_filter() from scipy . This is the code:

import numpy as np
import scipy.ndimage as ndimage

data = np.random.rand(400,128)
dimx = int(np.sqrt(np.size(data,0)))
dimy = dimx    
coord = np.random.randint(np.size(data,0), size=(dimx,dimy))

def test_func(values):
    idx_center = int(values[4])
    weight_center = data[idx_center]
    weights_around = data[values]
    differences = weights_around - weight_center
    distances = np.linalg.norm(differences, axis=1)
    return np.max(distances)

results = ndimage.generic_filter(coord,
                                 test_func,
                                 footprint = np.ones((3,3)))

When I execute it though, the following error shows up:

SystemError: <class 'int'> returned a result with an error set

when trying to coerce values[4] to an int . If I run the function test_func() without using ndimage.generic_filter() for a random array values , the function works alright.

Why is this error occurring? Is there a way to make it work?

Answer 1

For your case:

This must be a bug in either Python or SciPy. Please file a bug at https://bugs.python.org and/or https://www.scipy.org/bug-report.html . Include the version numbers of Python and NumPy/SciPy, the full code that you have here, and the entire traceback.

(Also, if you can find a way to trigger this bug that doesn't require the use of randomness, they will likely appreciate it. But if you can't find such a method, then please do file it as-is.)

In general:

"[R]eturned a result with an error set" is something that can only be done at the C level. In general, the Python/C API expects most C functions to do one of two things:

1. Set an exception using one of these functions and return NULL (corresponds to throwing an exception).
2. Don't set an exception and return a "real" value, usually a PyObject* (corresponds to returning a value, including returning None ).

These two cases are normally incorrect:

3. Set an exception (or fail to clear one that already exists), but then return some value other than NULL .
4. Don't set an exception, but then return NULL .

Python is raising a SystemError because the implementation of int , in the Python standard library, tried to do (3), possibly as a result of SciPy doing it first. This is always wrong, so there must be a bug in either Python or the SciPy code that it called into.

Answer 2

I was having a very similar experience with Python 3.8.1 and SciPy 1.4.1 on Linux. A workaround was to introduce np.floor so that:

centre = int(window.size / 2) becomes centre = int(np.floor(window.size/2))

which seems to have resolved the issue.