python os.read() not reading correct number of bytes

Question

I'm trying to read blocks from a binary file (oracle redo log) but I'm having a issue where, when I try to read a 512 byte block using os.read(fd,512) I am returned less than 512 bytes. (the amount differs depending on the block)

the documentation states that "at most n Bytes" so this makes sense that I'm getting less than expected. How can I force it to keep reading until I get the correct amount of bytes back?

I've attempted to adapt the method described here Python f.read not reading the correct number of bytes But I still have the problem

def read_exactly(fd, size):
    data = b''
    remaining = size
    while remaining:  # or simply "while remaining", if you'd like
        newdata = read(fd, remaining)
        if len(newdata) == 0:  # problem
            raise IOError("Failed to read enough data")
        data += newdata
        remaining -= len(newdata)
    return data


def get_one_block(fd, start, blocksize):
    lseek(fd, start, 0)
    blocksize = blocksize

    print('Blocksize: ' + str(blocksize))
    block = read_exactly(fd, blocksize)
    print('Actual Blocksize: ' + str(block.__sizeof__()))
    return block

which then returns the error: OSError: Failed to read enough data

My code:

from os import open, close, O_RDONLY, lseek, read, write, O_BINARY, O_CREAT, O_RDWR

def get_one_block(fd, start, blocksize):
    lseek(fd, start, 0)
    blocksize = blocksize

    print('Blocksize: ' + str(blocksize))
    block = read(fd, blocksize)
    print('Actual Blocksize: ' + str(block.__sizeof__()))

    return block

def main():
    filename = "redo_logs/redo03.log"
    fd = open(filename, O_RDONLY, O_BINARY)
    b = get_one_block(fd, 512, 512)

Output

Blocksize: 512
Actual Blocksize: 502

in this instance the last byte read is 0xB3 which is followed by 0x1A which i believe is the problem.

00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
EF 42 B8 5A DC D1 63 1B A3 31 C7 5E 9F 4A B7 F4 
4E 04 6B E8 B3<<-- stops here -->>1A 4F 3C BF C9 3C F6 9F C3 08 02 
05 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Any help would be greatly appreciated:)

Answer 1

You need to read inside a while loop and check the true number of bytes you've got.

If you got less you read again with the left delta.

the while exits when you got what you expected or reached EOF.