boost::property_tree::ptree and UTF-8 with BOM

Question

Is boost::property_tree::ptree can't handle files which use UTF-8 with BOM?

#include <boost/filesystem.hpp>
#include <boost/property_tree/ini_parser.hpp>

#include <cstdlib>
#include <iostream>

int main()
{
    try
    {
        boost::filesystem::path path("helper.ini");
        boost::property_tree::ptree pt;
        boost::property_tree::read_ini(path.string(), pt);
        const std::string foo = pt.get<std::string>("foo");
        std::cout << foo << '\n';
    }
    catch (const boost::property_tree::ini_parser_error& e)
    {
        std::cerr << "An error occurred while reading config file: " << e.what() << '\n';
        return EXIT_FAILURE;
    }
    catch (const boost::property_tree::ptree_bad_data& e)
    {
        std::cerr << "An error occurred while getting options from config file: " << e.what() << '\n';
        return EXIT_FAILURE;
    }
    catch (const boost::property_tree::ptree_bad_path& e)
    {
        std::cerr << "An error occurred while getting options from config file: " << e.what() << '\n';
        return EXIT_FAILURE;
    }
    catch (...)
    {
        std::cerr << "Unknown error \n";
        return EXIT_FAILURE;
    }
}

helper.ini

foo=str

Output

An error occurred while getting options from config file: No such node (foo)

What can i do with it? Manually delete BOM from file bedore reading it?

boost 1.53

Answer 1

I'm using this to skip BOM characters:

    boost::property_tree::ptree pt;
    std::ifstream file("file.ini", std::ios::in);
    if (file.is_open())
    {
        //skip BOM
        unsigned char buffer[8];
        buffer[0] = 255;
        while (file.good() && buffer[0] > 127)
            file.read((char *)buffer, 1);

        std::fpos_t pos = file.tellg();
        if (pos > 0)
            file.seekg(pos - 1);

        //parse rest stream
        boost::property_tree::ini_parser::read_ini(file, pt);

        file.close();
    }

Answer 2

1. Yes, easiest option would be to just check if the file starts with BOM and remove it.

2. You can file a bug against boost (probably should)

3. You could use boost::iosteams filters to remove BOM from the input stream whenever its found:

   • filter usage
   • filter concept
   • input filter concept with example that can be adapted to drop BOM