上下文无关文法微分整数和浮点常数

Question

I am writing an LR(1) parser, and I've been basing my test grammar off of the C language. I've looked at the grammar for both C and Python:

https://www.lysator.liu.se/c/ANSI-C-grammar-y.html https://docs.python.org/3/reference/grammar.html

C seems to use the symbol CONSTANT for integer and floating point constants, and Python uses NUMBER.

What I'm wondering is why are these not separated into individual symbols such as INT and FLOAT so that they can later be put into separate nodes in the Abstract Syntax Tree?

Since we already know what type of number it is after the lexer has parsed it, why merge them into a generic 'NUMBER' and later try to figure out which one it is again?

Answer 1

Being able to handle some special cases earlier does not simplify things, since you still need the same code in a different place later. For example, consider the code y + z . Python doesn't know what that is, other than at run time it will invoke y.__add__(z) . The code to generate that isn't going away. That same code can take 3 + x and just as easily generate (3).__add__(z) . So it doesn't really simplify anything to distinguish between y + z and 3 + z during parsing. (The same logic holds if y is a float literal instead of an identifier.)

Now consider something like 3.0 + 5 . Separate code exists to replace this with 8.0 instead of (3.0).__add__(5) prior to byte-code compilation, because 1) it's simple to do and 2) it is demonstrably better than invoking a function at run time. However, this still isn't done by the parser. This is done by an optimizer that runs over the tree looking for things like NUMBER + NUMBER . Once that is found, the optimizer can determine if the NUMBER s are ints or floats, and produce the appropriate sum to include in the code. This is simpler than having to handle 4 different bits of parse tree INT + FLOAT , FLOAT + INT , FLOAT + FLOAT , and INT + INT .