Check the maximum occurence of array in a nested function using ANTLR4

Question

I have the below grammar file for parsing nested functions.

Grammer:

grammar FunctionTokenizer;

parse     : function* EOF;
function  : ID '(' expr_list? ')';
expr_list : expr (',' expr)*;
expr      : expr subscript | function | STRING | NUMBER | ID;
subscript : '[' expr? ']';

STRING    : '"' ~'"'* '"';
NUMBER    : [0-9]+ ('.' [0-9]+)?;
ID        : [a-zA-Z_] [a-zA-Z_0-9]*;
SPACES    : [ \t\r\n]+ -> skip;

Input is failing as it has square bracket.I changed the regex for lexer as:

ID        : [a-zA-Z_] [a-zA-Z_0-9\\[\\]]*;

But it is giving error.

input:

split(mul(add(input["data"][0]["name"][]["node"],input["data"][0]),input["data"][0]["name"][]["node"][]),",")

For the above input I am checking which argument has maximum brackets either [](empty) or [0-9] square brackets with number neglecting those having single and double quotes like ["1234"] or ['data'] .

For the above input maximum array is: 3 because argument input["data"][0]["name"][]["node"][] is having maximum array.

So, I am trying to implement the visitor pattern implementation but the expression list is not reaching to visitExpr_list method.

Below is the GitHub link for the project:

https://github.com/VIKRAMAS/CheckMaxArrayInNestedFunction/tree/master

VisitorImplementation class:

public class FunctionValidateVisitorImpl  extends FunctionTokenizerBaseVisitor<String>  {


    @Override
    public String visitParse(FunctionTokenizerParser.ParseContext ctx) {
        ParseTree name = ctx.getChild(2);
        String visit = visit(name);
        System.err.println("visit1:::::::::::"+visit);
        return visit(name);
    }


    @Override
    public String visitFunction(FunctionTokenizerParser.FunctionContext ctx) {
        ParseTree name = ctx.getChild(2);
        String visit = visit(name);
        System.err.println("visit2:::::::::::"+visit);
        return visit;
    }


    @Override
    public String visitExpr_list(FunctionTokenizerParser.Expr_listContext ctx) {
        String s="";
        for (int i = 0; i < ctx.getChildCount(); i+=2) {
        if(ctx.getChild(i) instanceof FunctionTokenizerParser.Expr_listContext ) {
            String g=visit(ctx.getChild(i));
            System.err.println("visit3:::::::::::if "+i+"    "+g);
            s=s+g;
        }
        else {
            System.err.println("visit4:::::::::::else  "+i+"    "+ctx.getChild(i).getText());
            s=s+ctx.getChild(i).getText();
        }
        }
        return s;
    }


    public String visitSubscript(FunctionTokenizerParser.SubscriptContext ctx) {
        System.err.println("visit5:::::::::::");
        String s="";
        for (int i = 0; i < ctx.getChildCount(); i++) {
        if(ctx.getChild(i) instanceof FunctionTokenizerParser.SubscriptContext) {
            String g=visit(ctx.getChild(i));
            s=s+g;
        }
        else {
            s=s+ctx.getChild(i).getText();
        }
        }
        return s;
    }


    @Override
    public String visitExpr(FunctionTokenizerParser.ExprContext ctx) {
        System.err.println("visit6:::::::::::");
        String s="";
        for (int i = 0; i < ctx.getChildCount(); i++) {
            if(ctx.getChild(i) instanceof TerminalNodeImpl ) {
                s=s+ctx.getChild(i).getText();
            }
            else {
                String g=visit(ctx.getChild(i));

                s=s+g;
            }
        }

        return s;
    }



}

Test class:

public class FunctionValidate {


    public static void main(String[] args) {
        try {
            String input = "mul(add(input[\"data\"][0][\"name\"][][\"node\"],input[\"data\"][0]),input[\"data\"][0][\"name\"][][\"node\"][])";
            ANTLRInputStream str = new ANTLRInputStream(input);
            FunctionTokenizerLexer lexer = new FunctionTokenizerLexer(str);
            CommonTokenStream tokens = new CommonTokenStream(lexer);
            FunctionTokenizerParser parser = new FunctionTokenizerParser(tokens);
            parser.removeErrorListeners(); // remove ConsoleErrorListener 
            parser.addErrorListener(new VerboseListener());
             FunctionContext tree = parser.function();
            FunctionValidateVisitorImpl visitor = new FunctionValidateVisitorImpl();
            visitor.visit(tree);
            System.out.println("-->"+tree.toStringTree( parser ));
            AST ast=new AST(tree);
            System.out.println( "Improved ParseTree:\n" + ast.toString() );
            JFrame frame = new JFrame("Antlr AST");
            JPanel panel = new JPanel();
            TreeViewer viewr = new TreeViewer(Arrays.asList(
            parser.getRuleNames()),tree);
            viewr.setScale(1.5);
            panel.add(viewr);
            frame.add(panel);
            frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
            frame.setSize(500,500);
            frame.setVisible(true);
        } catch (Exception e) {
            e.printStackTrace();
        } 
    }



}

The visitor implimentation is working for one argument but if I pass multiple arguments it is failing. input:

split(rrr(test(input[\"data\"][0]),input[\"data\"]),input[\"data\"])[]

ouput:

visit4:::::::::::else  0    rrr(test(input["data"][0]),input["data"])
visit4:::::::::::else  2    input["data"]
visit2:::::::::::rrr(test(input["data"][0]),input["data"])input["data"]

After expression list method, directly else loop is executing but it should call visit method. Instance of expression list condition is failing so, the method call is directly reaching to else.

Answer 1

For input like bar["foo"] , ["foo"] is not something you want to glue to your bar in your lexer. This is something the parser should recognise as an expression.

How about something like this:

expr      : expr subscript | function | STRING | NUMBER | ID;
subscript : '[' expr? ']';

This would also match foo()[42] . If you only want to match identifiers preceded by subscript , change expr subscript to ID subscript .