Non-blocking file read

Question

How can I read the contents of a binary or a text file in a non-blocking mode?

For binary files: when I open(filename, mode='rb') , I get an instance of io.BufferedReader . The documentation fort io.BufferedReader.read says :

Read and return size bytes, or if size is not given or negative, until EOF or if the read call would block in non-blocking mode.

Obviously a straightforward open(filename, 'rb').read() is in a blocking mode. To my surprise, I could not find an explanation anywhere in the io docs of how to choose the non-blocking mode.

For text files: when I open(filename, mode='rt') , I get io.TextIOWrapper . I assume the relevant docs are those for read in its base class, io.TextIOBase ; and according to those docs , there seems no way to do non-blocking read at all:

Read and return at most size characters from the stream as a single str. If size is negative or None, reads until EOF.

Answer 1

File operations are blocking. There is no non-blocking mode.

But you can create a thread which reads the file in the background. In Python 3, concurrent.futures module can be useful here.

from concurrent.futures import ThreadPoolExecutor

def read_file(filename):
    with open(filename, 'rb') as f:
        return f.read()

executor = concurrent.futures.ThreadPoolExecutor(1)
future_file = executor.submit(read_file, 'C:\\Temp\\mocky.py')

# continue with other work

# later:

if future_file.done():
    file_contents = future_file.result()

Or, if you need a callback to be called when the operation is done:

def on_file_reading_finished(future_file):
    print(future_file.result())

future_file = executor.submit(read_file, 'C:\\Temp\\mocky.py')
future_file.add_done_callback(on_file_reading_finished)

# continue with other code while the file is loading...

Answer 2

I suggest using aiofiles - a library for handling local disk files in asyncio applications.

f = yield from aiofiles.open('filename', mode='r')
try:
    contents = yield from f.read()
finally:
    yield from f.close()
print(contents)
'My file contents'

async style version

async def read_without_blocking():
    f = await aiofiles.open('filename', mode='r')
    try:
        contents = await f.read()
    finally:
        await f.close()

Answer 3

Python does support non-blocking reads, at least on Unix type systems, by setting the O_NONBLOCK flag . In Python 3.5+, there is theos.set_blocking() function which makes this easier:

import os
f = open(filename, 'rb')
os.set_blocking(f.fileno(), False)
f.read()  # This will be non-blocking.

However, as zvone's answer notes, this doesn't necessarily work on actual disk files. This isn't a Python thing though, but an OS limitation. As the Linux open(2) man page states:

Note that this flag has no effect for regular files and block devices; that is, I/O operations will (briefly) block when device activity is required, regardless of whether O_NONBLOCK is set.

But it does suggest this may be implemented in the future:

Since O_NONBLOCK semantics might eventually be implemented, applications should not depend upon blocking behavior when specifying this flag for regular files and block devices.