Lex: Breaking up long regular expressions over multiple lines

Question

What is the correct syntax for breaking long lex regular expressions over multiple lines in a .l file.

For example, say I have a regular expression like:

word1|word2|word3|word4  ECHO;

When I attempt to do this:

word1|word2|
word3|word4  ECHO;

I get an error. What is the correct way for breaking up regex over multiple lines in lex?

Answer 1

With flex (as an extension to standard lex syntax), you can use the (?x:…) syntax, similar to PCRE/Perl extended syntax. Note that unlike PCRE, the text to which the x flag applies is surrounded by parentheses. [Note 1].

Within the parentheses, comments and whitespace are ignored unless they are escaped or quoted. So you can write:

(?x:
   word1 |
   word2 |
   word3 |
   word4 )    ECHO;

Note: This syntax cannot be used in the definitions section, only in the rules section. I don't know if that is by design or whether some future enhancement might lift the restriction.

See the flex manual for a few more details. (It's in the section which starts '(?rs:pattern)')

---

Notes

1. In PCRE (that is, python), you would write (?x) --- extended regex , and the extension continues until the end of the regex, unless you turn it off. I won't even try to explain the rules Perl uses to detect the end of an eXtended regex.