How to discard non-terminal in grammar file with PLY (Python Lex-Yacc)

Question

I have faced a problem when using PLY. I want to create a call graph generator by PLY. In some situation, I need to discard some tokens in the grammar file. That is because I need to do something when the parser recognize that token before I discard it, so I can't just discard in the lexer file. For example, the 'IF' token is the one which I want to discard. So I try to do something to discard it in the grammar file. Just like:

def p_if(p):
    'if : IF'
    print "if"
    parser.symstack.pop()

But things didn't go the way I think. I print the symstack(it's a atribute of parser, and parser is a LRParser instance of yacc.py), and the symstack list just contain the previous tokens but not 'if'. So I am wondering how to discard a token in this situation. Could anyone help me? Thanks a lot!

Answer 1

You asked this a while ago and yet no comments or answers. Here is a possible explanation why.

I'm looking at the documentation in http://www.dabeaz.com/ply/ply.html . There is nothing there about manipulating the parser.symstack. It does not appear to be an exposed, documented interface.

Ie you're sticking a fork into a toaster.

Don't you tink the parser might be confused if you mess with its internal data structure, or that it might break with the next version of the parser generator, etc? Also, you don't adequately explain what you're trying to do. What is the objective of discarding the nonterminal symbols, and what exactly does that mean? What are those situations and what is the problem? Maybe the problem can be solved without poking into the parser object.

Also, there isn't a documented global parser object. You must have defined this parser variable like this "parser = yacc.yacc()" (documented in Section 7: Multiple Parsers and Lexers). Since you then refer to this variable in the the parser rules themselves, that appears to be very kludgy. There is a proper way for rules to refer to the parser, and that is through the p parameter they receive: you want p.parser. (Documented in Section 7, again).

Answer 2

Thanks a lot!! Actually, I have found a way to solve my problem. You are right, the parser.symstack should never be manipulated. Because my English is not good(my mother language is not English), I didn't explain my problem clearly.But I still appreciate for your advice. The following is my solution, and I hope this will help someone else in the future.

I. define p_error(p) function

def p_error(p):
tok = yacc.token() # Get the next token
yacc.errok()
return tok #Return the next token

II. raise syntax error in the function you need

def p_somerule(p):
' a : A '
raise SyntaxError