[英]Represent regular expression as context free grammar
I am hand-writing a parser for a simple regular expression engine. 我正在为一个简单的正则表达式引擎编写解析器。
The engine supports a .. z
|
引擎支持
a .. z
|
*
and concatenation and parentheses *
以及串联和括号
Here is the CFG I made: 这是我制作的CFG:
exp = concat factor1
factor1 = "|" exp | e
concat = term factor2
factor2 = concat | e
term = element factor3
factor3 = * | e
element = (exp) | a .. z
which is equal to 等于
S = T X
X = "|" S | E
T = F Y
Y = T | E
F = U Z
Z = *| E
U = (S) | a .. z
For alternation and closure, I can easily handle them by looking ahead and choose a production based on the token. 对于交替和关闭,我可以轻松地处理它们,方法是向前看,然后根据令牌选择产品。 However, there is no way to handle concatenation by looking ahead cause it is implicit.
但是,由于它是隐式的,因此无法通过向前看来处理串联。
I am wondering how can I handle concatenation or is there anything wrong with my grammar? 我想知道如何处理串联或语法有问题吗?
And this is my OCaml code for parsing: 这是我的OCaml代码进行解析:
type regex =
| Closure of regex
| Char of char
| Concatenation of regex * regex
| Alternation of regex * regex
(*| Epsilon*)
exception IllegalExpression of string
type token =
| End
| Alphabet of char
| Star
| LParen
| RParen
| Pipe
let rec parse_S (l : token list) : (regex * token list) =
let (a1, l1) = parse_T l in
let (t, rest) = lookahead l1 in
match t with
| Pipe ->
let (a2, l2) = parse_S rest in
(Alternation (a1, a2), l2)
| _ -> (a1, l1)
and parse_T (l : token list) : (regex * token list) =
let (a1, l1) = parse_F l in
let (t, rest) = lookahead l1 in
match t with
| Alphabet c -> (Concatenation (a1, Char c), rest)
| LParen ->
(let (a, l1) = parse_S rest in
let (t1, l2) = lookahead l1 in
match t1 with
| RParen -> (Concatenation (a1, a), l2)
| _ -> raise (IllegalExpression "Unbalanced parentheses"))
| _ ->
let (a2, rest) = parse_T l1 in
(Concatenation (a1, a2), rest)
and parse_F (l : token list) : (regex * token list) =
let (a1, l1) = parse_U l in
let (t, rest) = lookahead l1 in
match t with
| Star -> (Closure a1, rest)
| _ -> (a1, l1)
and parse_U (l : token list) : (regex * token list) =
let (t, rest) = lookahead l in
match t with
| Alphabet c -> (Char c, rest)
| LParen ->
(let (a, l1) = parse_S rest in
let (t1, l2) = lookahead l1 in
match t1 with
| RParen -> (a, l2)
| _ -> raise (IllegalExpression "Unbalanced parentheses"))
| _ -> raise (IllegalExpression "Unknown token")
For a LL grammar the FIRST sets are the tokens that are allowed as first token for a rule. 对于LL语法,FIRST集是允许作为规则的第一个标记的标记。 To can construct them iteratively till you reach a fixed point.
可以迭代地构造它们,直到达到固定点为止。
Start with step 1 and then repeat steps 2 and 3 until the FIRST sets reach a fixed point (don't change). 从第1步开始,然后重复第2步和第3步,直到FIRST设置达到固定点(不要更改)。 Now you have the true FIRST sets for your grammar and can decide every rule using the lookahead.
现在,您已经有了语法的真正第一集,并且可以使用前瞻性来确定每个规则。
Note: In your code the parse_T function doesn't match the FIRST(T) set. 注意:在您的代码中,parse_T函数与FIRST(T)集不匹配。 If you look at for example 'a|b' then is enters parse_T and the 'a' is matched by the parse_F call.
例如,如果您查看“ a | b”,则输入parse_T,而“ a”与parse_F调用匹配。 The lookahead then is '|'
然后,前瞻为“ |” which matches epsilon in your grammar but not in your code.
在您的语法中匹配epsilon,但在您的代码中不匹配。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.