JAVACC tokens definition

Question

I am working on a project using javacc , I have a problem and need some help, I have something like this in a file :

STRING COPYRIGHT (C) 2003, 2004 SYNOPSYS, INC.;

I have a token defined for the word STRING and I want to define another token ( NAME ) to get all the words COPYRIGHT (C) 2003, 2004 SYNOPSYS, INC.

The problem is that I have a token INTEGER and when it reach the word 2003 it says ( There was an error during the parse. Encountered INTEGER : 2003 ...)

So how can I do to get the words between "STRING" and ";"

Thank you!

Answer 1

I'm going to assume that you want <NAME> to match all characters after "STRING" and before the ";" and also the semicolon itself

You can do this using lexical states. You can read about them in the FAQ , the description of the grammar file , and the token manager minitutorial .

In short the token manager productions look like this

TOKEN: { <STRING: "STRING" > : AFTERSTRING } // When "STRING" is encountered, switch to state AFTERSTRING
<AFTERSTRING> TOKEN : { <NAME: (~[";"])* ";" > : DEFAULT} // After the ";", return to the DEFAULT state.

And the parser production can then look like this:

void string() : {} { <STRING> <NAME> }

If the semicolon is missing the lexer will throw a TokenManagerError .

---

EDIT: To get a better error message in case of a missing semicolon you could do the following.

TOKEN: { <STRING: "STRING" > : AFTERSTRING } // When "STRING" is encountered, switch to state AFTERSTRING
<AFTERSTRING> TOKEN : { <NAME: (~[";"])* ";" > : DEFAULT} // After the ";", return to the DEFAULT state.
<AFTERSTRING> TOKEN : { <MISSING_SEMI: ~[] > : DEFAULT } 

And the parser production can then look like this:

void string() : {
    Token t ;
} {
    t = <STRING>
    (
        <NAME>
    |
        <MISSING_SEMI>
        {throw new ParserException( "STRING must be followed by a \";\". at line"
                                 +t.beginLine+ ", column " +t.beginColumn+ "." ) ; }
    )
}