Threaded port scanner: Too many open files (after closing socket)

Question

I wrote a little threaded port scanner, where I paid attention to closing the sockets after I'm done with them.

Despite closing the sockets, I'm getting Too many open files errors when I run it against a large number of target hosts (~ 12'000 hosts, and 5 ports, so a total of 60'000 sockets created/closed).

My understanding is that I should only encounter this error when closing the ports. I'm not really interested in the option that requires me to modify the number of open files my system can handle (recommend in other SO posts).

Full port scanner code below.

import socket
import itertools
from concurrent.futures import ThreadPoolExecutor, as_completed

def check_port(host, port):
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.settimeout(1)

    if sock.connect_ex((host, port)) == 0:
        sock.close()
        return host, port

    sock.close()
    return False

hosts = open('/tmp/hosts').read().splitlines()
ports = [80, 443, 8000, 8080, 8443]

with ThreadPoolExecutor(max_workers=100) as executor:
    future_to_host_port = [
        executor.submit(check_port, host, port)
        for host, port in itertools.product(hosts, ports)
    ]

    for future in as_completed(future_to_host_port):
        try:
            result = future.result()
            if not result:
                continue
            print(result)
        except socket.gaierror as e:
            print(f'couldn\'t resolve')
        except socket.timeout as e:
            print(f'socket timeout')
        except Exception as e:
            print(type(e))
            print(f'error {e}')

Answer 1

Not sure this is what's causing your specific problem, but your call to close() should be inside a finally block. From connect_ex 's docs:

Like connect(address), but return an error indicator instead of raising an exception for errors returned by the C-level connect() call ( other problems, such as “host not found,” can still raise exceptions ).

Seems to me, if you're scanning 12,000 hosts, some of them don't exist and you leak a socket (or 5) for each of them.

Try this:

def check_port(host, port):
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
        sock.settimeout(1)

        if sock.connect_ex((host, port)) == 0:
            return host, port

    return False