Can't catch multiple exceptions from python3/paramiko

Question

In python3, I'm using multiprocessing.Pool to run paramiko in parallel to connect to every ssh device on a subnet. I get a series of errors which I can't seem to trap.

I have put a variety of try/except statements here and none of them catch this error. I've even wrapped the Pool setup and statements with try/except but nothing changed there. All the paramiko stuff is in the try_login function, so it should be coming from there, but all of it is in a try/except, and there's even a generic except which should get everything.

I've also tried combining exceptions that cause this output.

#!/usr/bin/env python3

import paramiko
import socket
import ipaddress

network_address = '192.168.50.0/24'
username = ''
password = ''
timeout = 3
num_threads = 30

def trylogin(ipaddress):
    global username, password
    try:
        ssh = paramiko.SSHClient()
        ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
        ssh.connect(ipaddress, username=username, password=password, timeout=timeout)
        (stdin, stdout, stderr) = ssh.exec_command('cat /var/serial_number')
        imei = stdout.readline()
        if imei != '': print("[+] {}: Success! Found ".format(ipaddress))
        return [ipaddress, imei]
    except paramiko.AuthenticationException:
        print("[-] {}: Authentication Exception!".format(ipaddress))
    except paramiko.SSHException:
        print("[-] {}: SSH Exception!".format(ipaddress))
    except (paramiko.ssh_exception.SSHException, OSError, EOFError):
        pass
    except EOFError:
        pass
    except OSError:
        pass
    except paramiko.ssh_exception.NoValidConnectionsError:
        pass
    except paramiko.ssh_exception.SSHException:
        pass
    except Exception as e:
        print('caught a new fish')
        print(e)
        pass
    finally:
        try:
            ssh.close()
        except:
            pass
    return

network = ipaddress.ip_network(network_address)
for ipaddr in network.hosts():
    print('Checking {}'.format(str(ipaddr)))
    trylogin(str(ipaddr))

What happens when this runs is that most of what's supposed to happen does. But I get an error showing two exceptions which I thought were handled. First an OSError, then a paramiko.ssh_exception.SSHException. I don't understand why I can't trap these.

$ ./find_ssh.py
[-] 192.168.50.1 trying...
[-] 192.168.50.2 trying...
[-] 192.168.50.3 trying...
[-] 192.168.50.4 trying...
[-] 192.168.50.5 trying...
[-] 192.168.50.6 trying...
[-] 192.168.50.7 trying...
[-] 192.168.50.8 trying...
[-] 192.168.50.9 trying...
[-] 192.168.50.10 trying...
[-] 192.168.50.10: Authentication Exception!
[-] 192.168.50.11 trying...
[-] 192.168.50.12 trying...
[-] 192.168.50.13 trying...
[-] 192.168.50.14 trying...
[-] 192.168.50.14: SSH Exception!
[-] 192.168.50.15 trying...
Exception: Error reading SSH protocol banner[Errno 9] Bad file descriptor
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/paramiko/transport.py", line 2211, in _check_banner
    buf = self.packetizer.readline(timeout)
  File "/usr/local/lib/python3.5/dist-packages/paramiko/packet.py", line 380, in readline
    buf += self._read_timeout(timeout)
  File "/usr/local/lib/python3.5/dist-packages/paramiko/packet.py", line 607, in _read_timeout
    x = self.__socket.recv(128)
OSError: [Errno 9] Bad file descriptor

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/paramiko/transport.py", line 2039, in run
    self._check_banner()
  File "/usr/local/lib/python3.5/dist-packages/paramiko/transport.py", line 2216, in _check_banner
    "Error reading SSH protocol banner" + str(e)
paramiko.ssh_exception.SSHException: Error reading SSH protocol banner[Errno 9] Bad file descriptor

[-] 192.168.50.16 trying...
[-] 192.168.50.17 trying...
[-] 192.168.50.18 trying...
[-] 192.168.50.19 trying...
[-] 192.168.50.20 trying...

EDIT: revised to remove multiprocessing calls. It's slower but clearer.

Answer 1

This is an ugly workaround more than it's an answer, but it makes the error disappear which is useful.

Add the line:

paramiko.util.log_to_file("main_paramiko_log.txt", level = "INFO")

The effect of this is that a file is created which contains the outputs from paramiko. It is straightforward to change the level from INFO to WARN or ERR. This particular error series is now printed to the logfile as an error, and no longer is printed to the screen.

Apparently uncaught errors are common in paramiko, there are a variety of related issues posted on their github, which I didn't realize before posting this.