flex 生成的 c++ 解析器稳定吗？

Question

I all, I'm reading this book and it says that the generation of c++ lexer with flex is a buggy experimental feature.

The book was published in 2009 so I would like to know if the issue was fixed in the last 10 years.

Answer 1

You cannot use a split lexer (=tokenizer) and parser for C++, since nested templates brackets are allowed:

std::map<int,std::map<int,int>>

This is allowed since C++11 and cannot cleanly be handled with a split lexter/parser approach. The problem is the last token ( >> ), which a lexer (which has no knowledge about the grammar) would see as a right shift operator. Repairing this in the grammar is virtually impossible.

Using a parsing approach which does not differentiate between characters and tokens fixes this problem. This means not using flex fixes this problem (and many others).

Please note that flex was mainly invented to work around the rather strong limitations of the bison/yacc parsers which can only parse LALR1 languages with a lookahead of one. The approach is dead since two decades.

Do not use flex/bison/yacc. Use a recursive descent parser PEG (parsing expression grammar) parser instead, eg PEGTL ( https://github.com/taocpp/PEGTL ).