Reading\Writing Structured Binary File in C++

Question

i want to read\\write a binary file which has the following structure:

The file is composed by "RECORDS". Each "RECORD" has the following structure: I will use the first record as example

• (red)START byte: 0x5A (always 1 byte, fixed value 0x5A)
• (green) LENGTH bytes: 0x00 0x16 (always 2 bytes, value can change from "0x00 0x02" to "0xFF 0xFF")
• (blue) CONTENT: Number of Bytes indicated by the decimal value of LENGTH Field minus 2.

In this case LENGHT field value is 22 (0x00 0x16 converted to decimal), therefore the CONTENT will contain 20 (22 - 2) bytes. My goal is to read each record one by one, and write it to an output file. Actually i have a read function and write function (some pseudocode):

private void Read(BinaryReader binaryReader, BinaryWriter binaryWriter)
{
    byte START = 0x5A;
    int decimalLenght = 0;
    byte[] content = null;
    byte[] length = new byte[2];

    while (binaryReader.PeekChar() != -1)
    {
        //Check the first byte which should be equals to 0x5A
        if (binaryReader.ReadByte() != START)
        {
            throw new Exception("0x5A Expected");
        }

        //Extract the length field value
        length = binaryReader.ReadBytes(2);

        //Convert the length field to decimal
        int decimalLenght = GetLength(length);

        //Extract the content field value
        content = binaryReader.ReadBytes(decimalLenght - 2);

        //DO WORK
        //modifying the content

        //Writing the record
        Write(binaryWriter, content, length, START);
    }
}

private void Write(BinaryWriter binaryWriter, byte[] content, byte[] length, byte START)
{
    binaryWriter.Write(START);
    binaryWriter.Write(length);
    binaryWriter.Write(content);   
}

As you can see i have already write it for C#, but i don't really know how to it with C++. Can someone please point me in the right direction ?

Answer 1

You'll need to use an std::ifstream and open the file in binary mode ( std::ios_base::binary ).

peek is very similar but returns eof instead of -1 if no character could be extracted. And read will enable you to read a given amount of bytes into a value. Note that some types you are familiar with ( byte , type[] ) don't exist in C++ or work different. You can use std::vector for the latter, but you need to define byte yourself.

Answer 2

I guess I'd do something on this order:

struct record { 
    static const int start = '\x5a';
    std::vector<char> data; // you might prefer unsigned char.
};

std::istream &operator>>(std::istream &is, record &r) { 
    char ch;
    short len;

    is.get(ch);
    verify(ch == record::start);
    is.read((char *)&len, sizeof(len));
    r.data.resize(len);
    is.read(&r.data[0], len);
    return is;
}

std::ostream &operator<<(std::ostream &os, record const &r) { 
    os << record::start;
    short len = (short)r.data.size();
    os.write((char *)&len, sizeof(len));
    os.write(&r.data[0], len);
    return os;
}

To processing a file of records like you've shown in your Read (poor name for something that reads, processes, and writes data, BTW) let's start by defining a functor to process a single record from the file:

class process_record { 
    record operator()(record r) { 
        // code to process a single record goes here
        // it will take one record as input, and return the processed record.
    }
}

Then, to process a file, we'd use code something like this:

std::transform(std::istream_iterator<record>(infile),
               std::istream_iterator<record>(),
               std::ostream_iterator<record>(outfile, ""),
               process_record());

[Note: I've used C-style casts for brevity here, but in real code I'd probably use static_cast s instead.]

Answer 3

Ok, based on your answers i have make some experiment:

string sFile = "C:\Test.bin";
static const int START_BYTE = '\x5a';
char tempByte;

ifstream inputFile (sFile, ios::in);
inputFile.open( sFile, ios::binary );

while (!inputFile.eof()) 
{
    inputFile.get(temptByte);

    cout << "Value of Byte " << hex << static_cast<int>(tempByte) << " hexadecimal" << endl;
}

However, the output always look as: ffffffcc, or if not converted as -52.

If i got it right my code should read the file one byte at time, and print out the byte hex value. Am i wrong ?

Answer 4

Thank to you all this is the solution i was able to develope:

// TestCPP003.cpp : Defines the entry point for the console application.
//

#include "stdafx.h"

#include "boost\program_options.hpp"
namespace po = boost::program_options;

#include "Util.h"

using namespace std;

int main(int argc, char* argv[])
{
    po::options_description desc("Allowed options");

    desc.add_options()
        ("h", "produce help message")
        ("i", po::value<string>(), "input file")
        ("o", po::value<string>(), "output file");

    po::variables_map vm;
    po::store(po::parse_command_line(argc, argv, desc), vm);

    if (vm.count("h"))
    {
        cout << desc << endl;
        return 0;
    }

    po::notify(vm);

    if (vm.count("i"))
    {
        cout << vm["i"].as<string>() << "\n";
    }

    string sInputFile = vm["i"].as<string>();

    if (fileExists(sInputFile.c_str()))
    {
        cout << "file exists: <" <<  sInputFile << ">" << endl;
    }
    else
    {
        cout << "file not exists: <" <<  sInputFile << ">" << endl;
        cout << "RETURN CODE: 8" << endl;
        return 8;
    }

    string sOutputFile = vm["o"].as<string>();

    static const int START_BYTE = '\x5a';
    static const int AFP_RECORD_HEADER_SIZE = 1;
    static const int AFP_RECORD_LENGTH_SIZE = 2;    
    char * afpHeaderBlock = new char[1];
    char * afpLengthBlock;
    unsigned int afpRecordLength = 0;
    char * afpContentBlock;

    ifstream inputStream(sInputFile, ios::in|ios::binary);
    ofstream outputStream(sOutputFile, ios::out|ios::binary);

    if (inputStream.is_open() && outputStream.is_open())
    {
        while (inputStream.read(afpHeaderBlock, AFP_RECORD_HEADER_SIZE)) 
        {           
            //cout << ToHex(string(afpHeaderBlock, AFP_RECORD_HEADER_SIZE), true) << endl;

            if (START_BYTE == afpHeaderBlock[0])
            {
                cout << "0x5A Found!" << endl;
            }
            else
            {
                cout << "0x5A not Found! - AFP Error" << endl;
            }

            outputStream.write(afpHeaderBlock,  AFP_RECORD_HEADER_SIZE);

            afpLengthBlock = new char[AFP_RECORD_LENGTH_SIZE];
            afpRecordLength = 0;

            inputStream.read(afpLengthBlock, AFP_RECORD_LENGTH_SIZE);

            //cout << ToHex(string(afpLengthBlock, AFP_RECORD_LENGTH_SIZE), true) << endl;

            afpRecordLength = (afpRecordLength << 8) + static_cast<const unsigned char&>(afpLengthBlock[0]);
            afpRecordLength = (afpRecordLength << 8) + static_cast<const unsigned char&>(afpLengthBlock[1]);

            //cout << "AFP Record Length: " << afpRecordLength << endl;

            outputStream.write(afpLengthBlock,  AFP_RECORD_LENGTH_SIZE);

            afpContentBlock = new char[afpRecordLength - AFP_RECORD_LENGTH_SIZE];

            inputStream.read (afpContentBlock, afpRecordLength - AFP_RECORD_LENGTH_SIZE);

            outputStream.write(afpContentBlock,  afpRecordLength - AFP_RECORD_LENGTH_SIZE);
        }

        inputStream.close();
        outputStream.flush();
        outputStream.close();
    }

    cout << "RETURN CODE: 0" << endl;
    return 0;
}