使用Java创建词法分析器程序的想法

Question

我正在尝试使用java创建一个词法分析器程序。程序必须具有标记化的概念。我在编译器编程方面有初学者的知识。我知道互联网上存在很多词法生成器。我可以使用它们来测试自己的词法分析器输出。但是我需要做自己的词法分析器。任何人都可以提供一些最佳参考文献或文章或想法来开始我的工作吗？

Answer 1

"Compilers Principles, Techniques and Tools" by Aho Sethi and Ullman has a chapter on lexical analysers. It includes a lot of the theory on regular expressions and finite automata that are core to this problem domain.

Answer 2

I would try taking a look at the source code for some of the better ones out there. I have used Sablecc in the past. If you go to this page describing how to to set you your environment, there is a link to the source code for it. Antlr is also a really commonly used one. Here is the source code for it.

Also, The Dragon Book is really good.

As Suggested by SK-logic I am adding Modern Compiler Implementation as another option.