std::istream ends unexpectedly

Question

I am trying to read a binary file in chunks. The file is approximately 11 kB. The following code reads from the file only once and the gcount() function says that only 1015 characters (bytes) were read and the stream is „depleted“ and the while loop does not continue (although the buffer is filled correctly up to buffer[4095] ):

std::array<int64_t, 256>  HuffmanTree::get_frequencies(std::istream & stream) const
{
    const size_t buffer_size = 4096;
    unsigned char buffer[buffer_size];
    std::array<int64_t, 256> frequencies;
    frequencies.fill(0);

    while (stream)
    {
        stream.read((char *)buffer, buffer_size);
        std::streamsize bytes_read = stream.gcount();

        for (std::streamsize i = 0; i < bytes_read; i++)
        {
            frequencies[buffer[i]]++;
        }
    }

    return frequencies;
}

What causes this behaviour and how could I fix it?

Edit: The stream.read(...) is called only ONCE and gcount() returns 1015. But the buffer contains the first 4096 bytes of the file (also I am running on Windows - VS 2017)

Answer 1

I believe your problem comes when you've less than 4096 bytes to read, and you call stream.read()

I use the following function, which TRIES to read 4096, but tidies up when it fails - maybe it's of use?

// Tries to read num bytes from inRaw into buffer (space previously allocated).  When num bytes does not take
// us off the end of the file, then this does the simple obvious thing and returns true.  When num bytes takes
// us over the edge, but things are still sane (they wanted 100 bytes, only 60 bytes to go), then this fills
// as much as possible and leaves things in a nice clean state.  When there are zero bytes to go, this
// return false.
bool safeRead(
    std::ifstream& inRaw,
    char* buffer,
    uint16_t num
)
{
    auto before = inRaw.tellg();
    if (inRaw.read(buffer, num) && inRaw.good() && !inRaw.eof())
    {
        return true;
    }
    else
    {
        if (inRaw.gcount())
        {
            inRaw.clear();
            auto pos = before + std::streamoff(inRaw.gcount());
            inRaw.seekg(pos);
            return true;
        }
        else
        {
            return false;
        }
    }
}