memory mapping a text file of floats separated by space to a vector of floating point numbers

Question

I have a text file with several thousands of floating point numbers separated by a space.I was trying to use memory mapping to copy all those data from the text file to a vector of floating point numbers in visual studio 2010 using c++.Following is the code i made to read text file to memory.Code is just reading random numbers which makes no sense. Can any one help me fixing it and copying the data to a vector of floats

#include<boost\iostreams\device\mapped_file.hpp>
#include<iostream>
int main()
{
boost::iostreams::mapped_file_source file;
int numberofElements = 1000000;
int numberofBytes = numberofElements*sizeof(float);
file.open("ReplayTextFile.txt",numberofBytes);

if(file.is_open())
{
float* data = (float*)file.data();
for(int i = 0;i <numberofElements;i++)
std::cout<<data[i]<<", ";

file.close();
} else
{
std::cout<<std::cout<<" couldnt map the file"<<std::endl;
}
system("pause");
return 0;
}

Answer 1

This is basically looking at the underlying representation of the input text, taking sizeof(float) bytes of that and attempting to treat that as an actual float .

In a typical case, a float will be four bytes, so given input like 1.23456 , it'll take 1.23 , look at the underlying representation (typically 0x31, 0x23, 0x32, 0x33) and interpret that much as a float .

Then it'll take 456 (0x34, 0x35, 0x36, 0x20) and interpret that as a second float .

Obviously, you need to convert the digits of one number into one float, ignore the space, then convert the digits of the next number into the next float .

The easiest way to do that would be to open the file as a stream, then initialize a vector<float> from istream_iterator s initialized from the file:

std::ifstream in("ReplayTextFile.txt");

std::vector<float> floats { std::istream_iterator<float>(in),
                            std::istream_iterator<float>()};

Answer 2

First, a terminology quibble: you're not copying the data from the file into a vector of floats, you're memory-mapping the data into an array of floats.

Second, when you memory map a file, the contents of memory are literally the same as the contents of the file on disk. So if a file had the number 2.203 and nothing else inside of it, when you memory mapped it and tried to read element 0 as a float, you would be reading (assuming sizeof(float)==4 ) the bytes (in hex) 32 2e 32 30 . These would be interpreted as a float, which is not what you want.

Instead, you need to at some point process the input and convert string representations into the bytes that represent that number as floating point. You can do that by opening the file with ifstream and then using the >> operator to read into a float.

However, if you want the runtime efficiency that comes with memory-mapping a file, you likely don't want to parse floats every time you run your program. In that case, you need to first preprocess the file to convert it from a series of numbers as strings to instead contain the raw bytes that represent the floating point numbers you want.

In my code, I've used the function below to write out bytes into an ostream opened with ios_base::binary .

void writeFloat(std::ostream &out, float f) {
    char *pt = reinterpret_cast<char*>(&f);
    out.put(pt[0]);
    out.put(pt[1]);
    out.put(pt[2]);
    out.put(pt[3]);
}

Once you've prepared the file, you should be able to memory map and read data from it as your code is already.