如何使用正则表达式在LaTeX中查找嵌套标签

Question

I'm trying to extract theorems from LaTeX source with java. My code almost works, but one test case is failing – nested theorems.

@Test
public void testNestedTheorems() {
    String source = "\\begin{theorem}" +
                    "this is the outer theorem" +
                    "\\begin{theorem}" +
                    "this is the inner theorem" +
                    "\\end{theorem}" +
                    "\\end{theorem}";

    LatexTheoremProofExtractor extractor = new LatexTheoremProofExtractor(source);
    extractor.parse();

    ArrayList<String> theorems = extractor.getTheorems();
    assertNotNull(theorems);
    assertEquals(2, theorems.size()); // theorems.size() is 1
    assertEquals("this is the outer theorem", theorems.get(0)); 
    assertEquals("this is the inner theorem", theorems.get(1)); 
}

Here's my theorem extractor which is called by LatexTheoremProofExtractor#parse :

private void extractTheorems() {

    // If this has been called before, return
    if(theorems != null) {
        return;
    }

    theorems = new ArrayList<String>();

    final Matcher matcher = THEOREM_REGEX.matcher(source);

    // Add trimmed matches while you can find them
    while (matcher.find()) {
        theorems.add(matcher.group(1).trim());
    }   
}

and THEOREM_REGEX is defined as follows:

private static final Pattern THEOREM_REGEX = Pattern.compile(Pattern.quote("\\begin{theorem}")
                                                    + "(.+?)" + Pattern.quote("\\end{theorem}"));

Does anyone have recommendations for dealing with the nested tags?

Answer 1

If you only want to match doubly nested theorem s, you can write down an explicit regular expression for it. I guess it would look something like this.

Pattern.compile(
      Pattern.quote("\\begin{theorem}")
        + "("
            + "(.+?)"
            + Pattern.quote("\\begin{theorem}")
                + "(.+?)"
            + Pattern.quote("\\end{theorem}")
        + ")*"
     + Pattern.quote("\\end{theorem}"));

(This code should give you the idea but it is untested an probably does not work like written. This is not the point I want to make.)

You can continue this for triple-nesting and so forth for any fixed number of nesting you want. Needless to say that it will become rather awkward pretty soon.

However, if your goal is to match arbitrary deep nestings then it is simply impossible to do with regular expressions. The problem is that the language you want to match is not regular (but context-free). Context-free languages are strictly more powerful than regular languages and regular expressions can only match regular languages precisely. In general, you will need to construct a push-down automaton if you want to match a context-free language.

Further reading:

• Chomsky hierarchy
• What is meant by “Now you have two problems”?