Boost Spirit Parser with a vector of three strings compiling into a struct, adapt not working

Question

I´ma student and need to write a Parser in C++ with the Boost-Library.

Therefore I write a grammer in QI because I need to parse into a struct. So far, so good.

I will give you some example-code. I think it is easier than writing the whole program down.

Description: So first we take a txt-File and read it, then the parser goes over it, says "Parsing is ok!" and parse into the struct. Our output is the struct in the console.

That works fine, now to some code examples. Here you can see the Grammar in Boost Spirit QI:

subject %= lexeme[lit("Fach: ") >> +(char_("a-zA-Z"))   >> lit("\n")]; //works!

        dozent %= lexeme[lit("Dozent: ") >> +(char_("a-zA-Z")) >> lit("\n")];

        date %= lexeme[lit("Datum: ") >> digit >> digit >> lit("-") >> digit >> digit >> lit("-") >> digit >> digit >> digit >> digit >> lit("\n")];

        count %= lexeme[lit("Anzahl: ") >> +digit >> lit("\n")];

        points %= lexeme[+digit >> lit("\t")];

        mark %= lexeme[digit >> lit("\n")];

        matnumber %= lexeme[(digit >> digit >> digit >> punct >> digit >> digit >> digit) >> lit("\t")];

        student %= matnumber >> points >> mark;

        start %=  subject >> dozent >> date >> count >> student;

That works fine, the rule for student brings the problem that we have an element with three parts. Matnumber, Points, and mark. That you can imagine what I mean, here the TXT-File which we try to parse:

Subject: Physics
Dozent: Wayne
Datum: 20-10-2014
Anzahl: 20
729.888 33  5
185.363 35  5

The last two lines are the rule student. And in the txt-File we have more than these two lines.

That we can take these lines as "student" we wrote a vector in our struct with typedef:

typedef boost::fusion::vector<string, string, string> student_t;

then we will use it in our struct:

struct klausur
{
    string str_subject;
    string str_dozent;
    string str_date;
    string count;
    string matr_nr;
    string points;
    string mark;
    string ende;
    student_t student;

    void ToString()
    {
        cout << "Struct.Fach: " << str_subject << endl;
        cout << "Struct.Dozent: " << str_dozent << endl;
        cout << "Struct.Datum: " << str_date << endl;
        cout << "Struct.Anzahl: " << count << endl;
        cout << "Struct.Mat_Nr: " << matr_nr << endl;
        cout << "Struct.Punkte: " << points << endl;
        cout << "Struct.Note: " << mark << endl;
        cout << "Struct.Student<0>: " << vec::at_c<0>(student); 
        cout << "Struct.Student<1>: " << vec::at_c<1>(student);
        cout << "Struct.Student<2>: " << vec::at_c<2>(student);

    }
};

Then we have our BOOST_ADAPT_STRUCT like this:

BOOST_FUSION_ADAPT_STRUCT(
client::klausur,
(string, str_subject)
(string, str_dozent)
(string, str_date)
(string, count)
(string, matr_nr)
(string, points)
(string, mark)
(student_t, student)

)

You see we have the typedef down there.

And then we have our rules in the Grammar.

    qi::rule<Iterator, string(), ascii::space_type> subject;
    qi::rule<Iterator, string(), ascii::space_type> dozent;
    qi::rule<Iterator, string(), ascii::space_type> date;
    qi::rule<Iterator, string(), ascii::space_type> count;
    qi::rule<Iterator, string(), ascii::space_type> matnumber;
    qi::rule<Iterator, string(), ascii::space_type> points;
    qi::rule<Iterator, string(), ascii::space_type> mark;
    qi::rule<Iterator, boost::fusion::vector<boost::fusion::vector<std::string, std::string, std::string> >()> student; 

And there is the hopefully final problem for our project...

We don´t know which datatype the qi:rule needs that the BOOST_ADAPT... works fine with it. All the other points are strings, but don´t know how to implement the own vector we created.

All the other rules are working fine and are in the struct later just the vector makes problems.

Has somebody an idea about that? I can upload more files and code-snippets if you need, but I still think that it´s maybe just a small problem that I can´t see. I look around for many boost topics but did not found the right thing.

I have to add the info that I am just a beginner, so maybe I did not explain everything right and... yeah. Hope you understand it. Also my english is not the best...

Thank you in advance for your help.

William

Answer 1

Spirit is a parser generator. You don't seem to actually parse anything (you just "extract" character sequences, which is more like tokenizing).

I'd do this:

• use proper data types
• use blank for skipping (doesn't include eol )
• put the eol expectation in the right spot
• put the lexemes in the right spot
• make date_t it's own type
• make student_t it's own type
• FIX the rule to use std::vector<student_t>() instead of ~ fusion::vector<student_t>() (that was a mistake)
• use operator<< to do the printing
• use repeat(n) [ student >> eol ] to parse the expected number of student lines
• use qi::locals to actually pass the number of students expected into repeat()

Live On Coliru

#define BOOST_SPIRIT_DEBUG
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix.hpp>
#include <boost/fusion/adapted/struct.hpp>
#include <iostream>
#include <iomanip>

namespace qi = boost::spirit::qi;
namespace vec = boost::fusion;

struct student_t {
    std::string matr_nr;
    unsigned    points;
    int         mark;
};

struct date_t {
    unsigned dd, mm, yyyy;

    friend std::ostream& operator<<(std::ostream& os, date_t const& d) {
        std::ostream local(os.rdbuf());
        local << std::setw(2) << std::setfill('0') << d.dd <<
            "-" << std::setw(2) << std::setfill('0') << d.mm <<
            "-" << std::setw(4) << std::setfill('0') << d.yyyy;
        return os;
    }
};

BOOST_FUSION_ADAPT_STRUCT(student_t,
        (std::string,matr_nr)(unsigned,points)(int,mark))
BOOST_FUSION_ADAPT_STRUCT(date_t,
        (unsigned,dd)(unsigned,mm)(unsigned,yyyy))

struct klausur {
    std::string str_subject;
    std::string str_dozent;
    date_t date;

    unsigned count;
    std::vector<student_t>   students;

    friend std::ostream& operator<<(std::ostream& os, klausur const& k)
    {
        os << "Fach: "   << k.str_subject << '\n';
        os << "Dozent: " << k.str_dozent  << '\n';
        os << "Datum: "  << k.date        << '\n';
        os << "Anzahl: " << k.count       << '\n';
        for (auto& s : k.students) {
            os << "Mat_Nr: " << s.matr_nr << '\n';
            os << "Punkte: " << s.points  << '\n';
            os << "Note: "   << s.mark    << '\n';
        }
        return os;
    }
};

BOOST_FUSION_ADAPT_STRUCT(klausur,
        (std::string                     , str_subject)
        (std::string                     , str_dozent)
        (date_t                          , date)
        (unsigned                        , count)
        (std::vector<student_t>          , students)
    )

template <typename Iterator, typename Skipper = qi::ascii::blank_type>
struct grammar : qi::grammar<Iterator, klausur(), Skipper> {
    grammar() : grammar::base_type(start) {
        using namespace qi;
        subject   = "Fach:"   >> lexeme [ +~char_('\n') ] >> eol;
        dozent    = "Dozent:" >> lexeme [ +~char_('\n') ] >> eol;
        date      = "Datum:"  >> lexeme [uint_ >> '-' >> uint_ >> '-' >> uint_] >> eol;
        count     = "Anzahl:" >> uint_ >> eol;
        points    = uint_;
        mark      = int_parser<int, 10, 1, 1>(); // single base-10 digit

        // no clue about this format; what is it? Just a real number?
        matnumber = lexeme[digit >> digit >> digit >> punct >> digit >> digit >> digit];

        student   = matnumber >> points >> mark;

        _a_type expected;
        klausur_ %= subject
                 >> dozent
                 >> date
                 >> count            [ expected = _1 ]
                 >> repeat(expected) [ student >> (eol|eoi) ]
                 ;

        start     = klausur_;

        BOOST_SPIRIT_DEBUG_NODES((start)(klausur_)(student)(matnumber)(mark)(points)(count)(date)(dozent)(subject))
    }

  private:
    qi::rule<Iterator, klausur(), Skipper> start;
    qi::rule<Iterator, klausur(), Skipper, qi::locals<unsigned> > klausur_;

    qi::rule<Iterator, std::string()    , Skipper> subject;
    qi::rule<Iterator, std::string()    , Skipper> dozent;
    qi::rule<Iterator, date_t(),          Skipper> date;
    qi::rule<Iterator, unsigned()       , Skipper> count;
    qi::rule<Iterator, std::string()    , Skipper> matnumber;
    qi::rule<Iterator, unsigned()       , Skipper> points;
    qi::rule<Iterator, int()            , Skipper> mark;
    qi::rule<Iterator, student_t()      , Skipper> student;
};

int main() {
    using It = std::string::const_iterator;
    std::string const input =
R"(Fach: Physics
Dozent: Wayne
Datum: 20-10-2014
Anzahl: 2
729.888 33  5
185.363 35  5)";

    It f = input.begin(), l = input.end();

    grammar<It> g;
    klausur k;
    bool ok = qi::phrase_parse(f, l, g, qi::ascii::blank, k);

    if (ok) {
        std::cout << "Parse success\n";
        std::cout << k;
    } else {
        std::cout << "Parse failed\n";
    }

    if (f!=l) {
        std::cout << "Remaining input: '" << std::string(f,l) << "'\n";
    }
}

Output:

Parse success
Fach: Physics
Dozent: Wayne
Datum: 20-10-2014
Anzahl: 2
Mat_Nr: 729.888
Punkte: 33
Note: 5
Mat_Nr: 185.363
Punkte: 35
Note: 5

As well as 72 lines of debug output:

<start>
  <try>Fach: Physics\nDozent</try>
  <klausur_>
    <try>Fach: Physics\nDozent</try>
    <subject>
      <try>Fach: Physics\nDozent</try>
      <success>Dozent: Wayne\nDatum:</success>
      <attributes>[[P, h, y, s, i, c, s]]</attributes>
    </subject>
    <dozent>
      <try>Dozent: Wayne\nDatum:</try>
      <success>Datum: 20-10-2014\nAn</success>
      <attributes>[[W, a, y, n, e]]</attributes>
    </dozent>
    <date>
      <try>Datum: 20-10-2014\nAn</try>
      <success>Anzahl: 2\n729.888 33</success>
      <attributes>[[20, 10, 2014]]</attributes>
    </date>
    <count>
      <try>Anzahl: 2\n729.888 33</try>
      <success>729.888 33  5\n185.36</success>
      <attributes>[2]</attributes>
    </count>
    <student>
      <try>729.888 33  5\n185.36</try>
      <matnumber>
        <try>729.888 33  5\n185.36</try>
        <success> 33  5\n185.363 35  5</success>
        <attributes>[[7, 2, 9, ., 8, 8, 8]]</attributes>
      </matnumber>
      <points>
        <try> 33  5\n185.363 35  5</try>
        <success>  5\n185.363 35  5</success>
        <attributes>[33]</attributes>
      </points>
      <mark>
        <try>  5\n185.363 35  5</try>
        <success>\n185.363 35  5</success>
        <attributes>[5]</attributes>
      </mark>
      <success>\n185.363 35  5</success>
      <attributes>[[[7, 2, 9, ., 8, 8, 8], 33, 5]]</attributes>
    </student>
    <student>
      <try>185.363 35  5</try>
      <matnumber>
        <try>185.363 35  5</try>
        <success> 35  5</success>
        <attributes>[[1, 8, 5, ., 3, 6, 3]]</attributes>
      </matnumber>
      <points>
        <try> 35  5</try>
        <success>  5</success>
        <attributes>[35]</attributes>
      </points>
      <mark>
        <try>  5</try>
        <success></success>
        <attributes>[5]</attributes>
      </mark>
      <success></success>
      <attributes>[[[1, 8, 5, ., 3, 6, 3], 35, 5]]</attributes>
    </student>
    <success></success>
    <attributes>[[[P, h, y, s, i, c, s], [W, a, y, n, e], [20, 10, 2014], 2, [[[7, 2, 9, ., 8, 8, 8], 33, 5], [[1, 8, 5, ., 3, 6, 3], 35, 5]]]]</attributes><locals>(2)</locals>
  </klausur_>
  <success></success>
  <attributes>[[[P, h, y, s, i, c, s], [W, a, y, n, e], [20, 10, 2014], 2, [[[7, 2, 9, ., 8, 8, 8], 33, 5], [[1, 8, 5, ., 3, 6, 3], 35, 5]]]]</attributes>
</start>