使用带有错误处理的 `with Pool() as p`

Question

I've got a function being assigned to a multiprocessing pool :

with Pool(os.cpu_count() - 1) as p:
    N = len(fpaths)
    p.starmap(resample, zip(fpaths, new_paths, [sr] * N, ['WAV'] * N, [manifest] * N))

It's altering some audio files. One or two files are corrupted and have a sampling rate of zero. This is causing a divide by zero error, which looks like this:

multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 47, in starmapstar
    return list(itertools.starmap(args[0], args[1]))
  File "/home/jupyter/jn-kaggle/birdsong/who-said-what/wsw/preprocessing.py", line 35, in resample
    audio, sr = librosa.load(old_path, sr=sr)
  File "/opt/conda/lib/python3.7/site-packages/librosa/core/audio.py", line 172, in load
    y = resample(y, sr_native, sr, res_type=res_type)
  File "/opt/conda/lib/python3.7/site-packages/librosa/core/audio.py", line 553, in resample
    ratio = float(target_sr) / orig_sr
ZeroDivisionError: float division by zero
"""

I'd like to handle this error, however, what I've tried doesn't seem to be working:

def resample(old_path, new_path, sr, ext='WAV', manifest=None):
    try:
        with warnings.catch_warnings():
            warnings.simplefilter("ignore", UserWarning)
            print(f'Loading {old_path}',)
            audio, sr = librosa.load(old_path, sr=sr)
    except ZeroDivisionError:
        audio, sr = librosa.load(old_path, sr=None)
        print(f'Error loading file at {old_path}. file=sys.stderr)

I know that the ZeroDivisionError is directly caused by the librosa.load() function, which is getting an incorrect sampling rate of 0 from one of its own dependencies. I have a fix for it, but I need to catch the error. How do I do that?

Answer 1

This is not the final answer, but I don't want to squeeze a lot of code in a comment:

Make the exception general and take out the retry. I want to be sure the except block is called.

Try this code:

def resample(old_path, new_path, sr, ext='WAV', manifest=None):
    try:
        print("Start Resample: ", old_path)
        with warnings.catch_warnings():
            warnings.simplefilter("ignore", UserWarning)
            print(f'Loading {old_path}',)
            audio, sr = librosa.load(old_path, sr=sr) # maybe error here
        print("Done Resample: ", old_path)
    except Exception as ex:  # any exception
        # audio, sr = librosa.load(old_path, sr=None) # remove for now
        print('Error: ', ex)