如何从Java中的给定字符串中提取单词

Question

I am trying to extract all the words (even the words with brackets next to it - methods/functions in a programming language)

But I can only get the first word, not all the words. How can I iterate through all the words that match the given regex ?

Here is what I tried. My String is a text file I am reading and it looks like this.

infile >> name; 

infile >> Id;
cout << name << " " << Id << endl;
hwp = compute_hw_participation (infile);
tests = compute_tests(tests, infile);
totalscore = compute_totalscore (totalscore, infile);

printRecord (name, Id, hwp, tests, totalscore, outfile);
infile >> name; 

return 0;
}

Additionally, I am trying to find the methods in this String methods are

compute_hw_participation(infile)

compute_totalscore(totalscore, infile)

printRecord (name, Id, hwp, tests, total score, outfile) //this method has a space between the method name and parenthesis, I need to get the parenthesis too (until the end of the parenthesis) despite the space, how can I achieve that too?

This is what I have tried:

package com.codeingrams.recursion;

import java.util.regex.Matcher;
import java.util.regex.Pattern;

/**
 *
 * @author Jananath Banuka
 */
public class Test {

    private static final Pattern p = Pattern.compile(" [^\\s(]+\\([^)]*\\)|\\S+");

    public static void main(String[] args) {
        String text = "\n"
                + "compute_hw_participation(infile) infile >> name; \n"
                + "while(!infile.eof())\n"
                + "{\n"
                + "infile >> Id;\n"
                + "cout << name << \" \" << Id << endl;\n"
                + "hwp = compute_hw_participation (infile);\n"
                + "tests = compute_tests(tests, infile);\n"
                + "totalscore = compute_totalscore (totalscore, infile);\n"
                + "// grade\n"
                + "printRecord (name, Id, hwp, tests, totalscore, outfile);\n"
                + "infile >> name; \n"
                + "}\n"
                + "\n"
                + "return 0;\n"
                + "}\n"
                + "";

        // create matcher for pattern p and given string
        Matcher m = p.matcher(text);        
        // if an occurrence if a pattern was found in a given string...
        if (m.find()) {
            // ...then you can use group() methods.
            System.out.println(m.group(0)); // gives only infile                        
            System.out.println(m.group(1)); //this gives error arrayIndexoutofBound
        }

    }
}

Output:

compute_hw_participation(infile) Error: Exception in thread "main" java.lang.IndexOutOfBoundsException: No group 1 at java.util.regex.Matcher.group(Matcher.java:538) at com.codeingrams.recursion.Test.main(Test.java:44)

Answer 1

You need a pattern that matches function calls, ie, a name, possibly space, an opening parenthesis, some arguments, and a closing parenthesis.

Looking at the Javadoc for Pattern you see the character classes you can use in regular expressions. You'll need:

• letters or digits or underscores: \\w
• maybe white space: \\s* , the * means zero to many times
• an opening parenthesis, which you need to escape with a backslash as it has a special meaning in regular expressions: \\(
• some (or none) characters, until you find a closing parenthesis: [^)]* The [ and ] create a group, the ^ is negation, meaning anything but what's in the group.
• the actual closing parenthesis: \\)

Then you need to add another backlash to each backslash, as Java Strings also use the backslash for special characters like \\n .

You also need to add parenthesis for capturing around the data you're interested in. That is also the reason you had to quote the parenthesis to match them --- unquoted parenthesis means grouping or capturing.

With that the total regular expression is (\\w+\\s*\\([^)]*\\)) .

Here's the full program:

import java.util.regex.Matcher;
import java.util.regex.Pattern;

public class Test {
    public static void main(String[] args) {
        String text = "\n"
                + "compute_hw_participation(infile) infile >> name; \n"
                + "while(!infile.eof())\n"
                + "{\n"
                + "infile >> Id;\n"
                + "cout << name << \" \" << Id << endl;\n"
                + "hwp = compute_hw_participation (infile);\n"
                + "tests = compute_tests(tests, infile);\n"
                + "totalscore = compute_totalscore (totalscore, infile);\n"
                + "// grade\n"
                + "printRecord (name, Id, hwp, tests, totalscore, outfile);\n"
                + "infile >> name; \n"
                + "}\n"
                + "\n"
                + "return 0;\n"
                + "}\n";
        Pattern p = Pattern.compile("(\\w+\\s*\\([^)]*\\))");
        Matcher m = p.matcher(text);
        while (m.find()) {
            System.out.println(m.group());
        }
    }
}

You'll see the limitations of this simplistic approach very quickly: it also thinks while(!infile.eo() is a function as it looks like a function call. The code does not know about any possible language keywords. Note also that it does not catch the last closing parenthesis in the while expression. That is because it does not count parenthesis and just stops at the first closing parenthesis. The regex also has no clue about comments or Strings, and would happily pick up commented out code or Strings like "foo()" .

Because of that you're almost always better off using a real parser for the language you are dealing with.