Dropping a token in Yacc/Bison when an error is matched in the rhs production

Question

I am writing a simple calculator in which an expression can be reduced to a statement, or list of statements. If a bad expression triggers a syntax error, I try to catch it with a production rule and then ignore it by giving the rule no actions. However, I believe this still reduces to a stmt_list despite it not being a valid statement.

Is there a way to make it simply ignore tokens matched with an error and prevent them from being reduced and used later?

The code below matches error ';' and reduces it to a stmt_list . It will then try to reduce that stmt_list with a valid expression, but since the first production was never called this will trigger a memory exception. My objective is to have Bison literally do nothing if an error is matched, such that a later valid expression can be the first reduction to stmt_list .

stmt_list:
        expr ';'                {
                                    // Allocate memory for statement array
                                    stmts = (float*)malloc(24 * sizeof(float));
                                    // Assign pointer for array
                                    $$ = stmts;
                                    // At pointer, assign statement value
                                    *$$ = $1; 
                                    // Increment pointer (for next job)
                                    $$++;
                                }
        | stmt_list expr ';'    {
                                    $$ = $1;
                                    *$$ = $2;
                                    $$++;
                                }
        | error ';'             { } // Do nothing (ignore bad stmt)
        | stmt_list error ';'   { } // Do nothing (ignore bad stmt)
        ;

Answer 1

If you supply no action for a rule, bison/yacc provides the default action $$ = $1 .

In fact, you are not providing no action. You are providing an explicit action which does nothing. As it happens, if you use the C template, the parser will still perform the default action. In other templates, an action which does not assign a value to $$ might provoke a warning during parser generation. But it certainly won't modify your data structures so as to nullify the action. It can't know what that means. If you know, you should write it as the action:-).

It's not 100% clear to me why you are keeping the results of the evaluations in a fixed-size dynamically-allocated array. You make no attempt to detect when the array fills up, so it's entirely possible that you'll end up overflowing the allocation and overwriting random memory. Moreover, using a global like this isn't usually a good idea because it prevents you from building more than one list at the same time. (For example, if you wanted to implement function calls, since a function's arguments are also a list of expressions.)

On the whole, it's better to put the implementation of the expanding expression list in a simple API which is implemented elsewhere. Here, I'm going to assume that you've done that; for specificity, I'll assume the following API (although it's just one example):

/* The list header structure, which contain all the information
 * necessary to use the list. The forward declaration makes it
 * possible to use pointers to ExprList objects without having to
 * expose its implementation details.
 */
typedef struct ExprList ExprList;

/* Creates a new empty expression-list and returns a pointer to its header. */
ExprList* expr_list_create(void);

/* Resizes the expression list to the supplied size. If the list
 * currently has fewer elements, new elements with default values are
 * added at the end. If it currently has more elements, the excess
 * ones are discarded. Calling with size 0 empties the list (but
 * doesn't delete it).
 */
int expr_list_resize(ExprList* list, int new_length);

/* Frees all storage associated with the expression list. The
 * argument must have been created with expr_list_create, and its
 * value must not be used again after this function returns.
 */
void expr_list_free(ExprList* list);

/* Adds one element to the end of the expression-list.
 * I kept the float datatype for expression values, although I
 * strongly believe that its not ideal. But an advantage of using an
 * API like this is that it is easier to change.
 */
void expr_list_push(ExprList* list, float value);

/* Returns the number of elements in the expression-list. */
int expr_list_len(ExprList* list);

/* Returns the address of the element in the expression list
 * with the given index. If the index is out of range, behaviour
 * is undefined; a debugging implementation will report an error.
 */
float* expr_list_at(ExprList* list, int index);

With that API, we can rewrite the productions for valid expressions:

stmt_list: expr ';'            { $$ = expr_list_create();
                                 expr_list_push($$, $1);
                               }
         | stmt_list expr ';'  { $$ = $1;
                                 expr_list_push($$, $1);
                               }

Now for the error cases. You have two error rules; one triggers when the error is at the beginning of a list, and the other when the error is encountered after one or more (possibly erroneous) expressions have been handled. Both of these are productions for stmt_list so they must have the same value type as stmt_list does ( ExprList* ). Thus, they must do whatever you think is appropriate when a syntax error is produced.

The first one, when the error is at the start of the list, only needs to create an empty list. It's hard to see what else it could do.

stmt_list: error ';'           { $$ = expr_list_create(); }

It seems to me that there are at least two alternatives for the other error action, when an error is detected after the list has at least one successfully-computed value. One possibility is to ditch the erroneous item, leaving the rest of the list intact. This requires only the default action:

stmt_list: stmt_list error ';'

(Of course, you could add the action { $$ = $1; } explicitly, if you wanted to.)

The other possibility is to empty the entire list, so as to start from scratch with the next element:

stmt_list: stmt_list error ';' { $$ = $1;
                                 expr_list_resize($$, 0);
                               }

There are undoubtedly other possibilities. As I said, bison cannot figure out what it is that you intended (and neither can I, really). You'll have to implement whatever behaviour you want.