在Bison错误后如何跳过一行中的其余令牌

Question

I'm writing an application for an assignment that uses Flex and Bison to determine if a statement is valid or not. After detecting an error in a statement, I want to print an error message and move onto the next line to look at the next statement, but everything I'm trying doesn't work.

Researching online, Bison has a built in error token that can be used for error handling. By using error '\\n' {yyerrok;}, I should be able to achieve what I want, but it isn't working.

My Flex code:

%{
  #include <cstdio>
  #include <iostream>
  using namespace std;

  #include "exp.tab.h"  // to get the token types from Bison

%}
%%

--.*                    ;
[a-zA-Z][a-zA-Z0-9]*    {yylval.print = strdup(yytext); return ID;}
;\s*                    {return EOL;}
[-+*/%]                 {yylval.print = strdup(yytext); return OP;}
=                       {return EQU;}
\(                      {return OPEN;}
\)                      {return CLOSE;}
[0-9]                   ;
\n                      ;
\r                      ;
.                       ;
%%

My Bison tokens and rules:

%union{

    char *print;

}

%token EQU
%token <print> ID
%token EOL
%token <print> OP
%token OPEN
%token CLOSE

%%

lines: line
    |   lines line
;

line: ass {cout << " VALID" << endl;}
    |   exp {cout << " VALID" << endl;}
    |   error '\n' {yyerrok;}
;

ass: id EQU {cout << " ="; } exp EOL {cout << ";";}
;

exp: term
    |   exp op term 
;

term: id 
    |   OPEN {cout << "(";} exp op term CLOSE {cout << ")";}
;

id: ID {cout << $1; }

op: OP {cout << $1; }


%%

My yyerror() just prints "Error ".

My input for parsing:

-- Good (valid) statements:

first = one1 + two2 - three3 / four4 ;
second = one1 * (two2 * three3) ;
one1 * i8766e98e + bignum
second = (one1 * two2) * three3 ;
third = ONE + twenty - three3 ;
third = old * thirty2 / b567 ;

-- Bad (invalid) statements:

first = = one1 + two2 - three3 / four4 ;
first = one1 + - two2 - three3 / four4 ;
first = one1 + two2 - three3 / four4
first = one1 + two2 ? three3 / four4 ;
second = 4 + ( one1 * two2 ) * ( three3 + four4 ;
third = one1 + 24 - three3 ;
one1 +- delta
sixty6 / min = fourth ;

I'd expect the output to print an error then move to the next line

first =one1+two2-three3/four4; VALID
second =one1*(two2*three3); VALID
one1*i8766e98e+bignum VALID
second =(one1*two2)*three3; VALID
third =ONE+twenty-three3; VALID
third =old*thirty2/b567; VALID
first = Error
first = one1 + Error
first = one1 + two2 - three3 / four4 Error
first = one1 + two2 Error
.
.
.

But when I run it, it just stops at the first Error printing

first =one1+two2-three3/four4; VALID
second =one1*(two2*three3); VALID
one1*i8766e98e+bignum VALID
second =(one1*two2)*three3; VALID
third =ONE+twenty-three3; VALID
third =old*thirty2/b567; VALID
first = Error

Any help would be appreciated, but mostly I want to know why the error '\\n' rule isn't working and what I can do to fix it.

Answer 1

Since your lexer ignores \\n , telling the parser to skip tokens until it sees a newline will cause it to skip the rest of the file.

However, you can (almost) make this work by getting the lexer to recognise newline characters, but only during error recovery. (Check in the action for \\n and either ignore it or send it.)

But that will occasionally produce odd results, because the token which produces the error might be on the next line, in which case the newline will already have been ignored before the error was detected. For example, here the problem is a missing semicolon:

a = 1
while (a > 0) {
    …

But that error will only be detected after while has been read. (If the next token were, say, + , the parse should continue.) So skipping to the end of the line means continuing the parse at the third line, thus introducing an unbalanced brace.

Even so, it could be an interesting start.

Answer 2

Using '\\n' isn't working because your lexer never returns '\\n' , so there will never be any '\\n' tokens in the token stream. Basically if the lexer ignores certain characters, you can't use them in the parser in any way, including for error recovery.

So your two options would be to stop ignoring line breaks (probably a bad idea because then you'd have to mention them everywhere in the grammar where you want to allow line breaks) or to use some other token for error recovery. Skipping everything until the next semicolon would probably be a good alternative (though that still won't produce your expected output since not all of your lines end with semicolons).