Parsing DECAF grammar in ANTLR

Question

I am creating a the parser for DECAF with Antlr grammar DECAF ;

//********* LEXER ******************
LETTER: ('a'..'z'|'A'..'Z') ;
DIGIT : '0'..'9' ;
ID : LETTER( LETTER | DIGIT)* ;
NUM: DIGIT(DIGIT)* ;
COMMENTS: '//' ~('\r' | '\n' )*  -> channel(HIDDEN);
WS : [ \t\r\n\f | ' '| '\r' | '\n' | '\t']+  ->channel(HIDDEN); 

CHAR: (LETTER|DIGIT|' '| '!' | '"' | '#' | '$' | '%' | '&' | '\'' | '(' | ')' | '*' | '+' 

| ',' | '-' | '.' | '/' | ':' | ';' | '<' | '=' | '>' | '?' | '@' | '[' | '\\' | ']' | '^' | '_' | '`'| '{' | '|' | '}' | '~' 
'\t'| '\n' | '\"' | '\'');


// ********** PARSER *****************

program : 'class' 'Program' '{' (declaration)* '}'  ;

declaration: structDeclaration| varDeclaration | methodDeclaration  ;

varDeclaration: varType ID ';' | varType ID '[' NUM ']' ';'  ;

structDeclaration : 'struct' ID '{' (varDeclaration)* '}'  ;

varType: 'int' | 'char' | 'boolean' | 'struct' ID | structDeclaration | 'void'  ;

methodDeclaration : methodType ID '(' (parameter (',' parameter)*)* ')' block  ;

methodType : 'int' | 'char' | 'boolean' | 'void' ;

parameter : parameterType ID | parameterType ID '[' ']' ;

parameterType: 'int' | 'char' | 'boolean'  ;

block : '{' (varDeclaration)* (statement)* '}' ;

statement : 'if' '(' expression ')' block ( 'else' block )? 
           | 'while' '(' expression ')' block
           |'return' expressionA ';' 
           | methodCall ';' 
           | block  
           | location '=' expression 
           | (expression)? ';'  ;

expressionA: expression | ;


location : (ID|ID '[' expression ']') ('.' location)?  ;

expression : location | methodCall | literal | expression op expression | '-' expression | '!' expression | '('expression')'  ;

methodCall :    ID '(' arg1 ')' ;

arg1    :   arg2 | ;

arg2    :   (arg) (',' arg)* ;

arg :   expression;

op: arith_op | rel_op | eq_op | cond_op  ;

arith_op : '+' | '-' | '*' | '/' | '%' ;

rel_op : '<' | '>' | '<=' | '>=' ;

eq_op : '==' | '!=' ;

cond_op : '&&' | '||' ;

literal : int_literal | char_literal | bool_literal ;

int_literal : NUM ;

char_literal : '\'' CHAR '\'' ;

bool_literal : 'true' | 'false' ;

When I give it the input:

    class Program {

    void main(){

        return 3+5 ;
    }
    }

The parse tree is not building correctly since it is not recognizing the 3+5 as an expression. Is there anything wrong with my grammar that is causing the problem?

Answer 1

Lexer rules are matched from top to bottom. When 2 or more lexer rules match the same amount of characters, the one defined first will win . Because of that, a single digit integer will get matched as a DIGIT instead of a NUM .

Try parsing the following instead:

class Program {
    void main(){    
        return 33 + 55 ;
    }
}

which will be parsed just fine. This is because 33 and 55 are matched as NUM s, because NUM can now match 2 characters ( DIGIT only 1, so NUM wins ).

To fix it, make DIGIT a fragment (and LETTER as well):

fragment LETTER: ('a'..'z'|'A'..'Z') ;
fragment DIGIT : '0'..'9' ;
ID : LETTER( LETTER | DIGIT)* ;
NUM: DIGIT(DIGIT)* ;

Lexer fragments are only used internally by other lexer rules, and will never become tokens of their own.

A couple of other things: your WS rule matches way too much (it now also matches a | and a ' ), it should be:

WS : [ \t\r\n\f]+  ->channel(HIDDEN);

and you shouldn't match a char literal in your parser: do it in the lexer:

CHAR : '\'' ( ~['\r\n\\] | '\\' ['\\] ) '\'';

If you don't, the following will not get parsed properly:

class Program {
    void main(){
        return '1';
    }
}

because the 1 wil be tokenized as a NUM and not as a CHAR .