Java：在两个字符串之间匹配令牌并返回匹配的令牌数

Question

Need some help to find the number of matched tokens between two strings. I have a list of string stored in ArrayList (example given below):

Line 0 : WRB VBD NN VB IN CC RB VBP NNP  
Line 1 : WDT NNS VBD DT NN NNP NNP  
Line 2 : WRB MD PRP VB DT NN IN NNS POS JJ NNS  
Line 3 : WDT NN VBZ DT NN IN DT JJ NN IN DT NNP  
Line 4 : WP VBZ DT JJ NN IN  NN  

Here, you can see each string consists of a bunch of tokens separated by spaces. So, there's three things I need to work with..

1. Compare the first token (WRB) in Line 0 to the tokens in Line 1 to see if they match. Move on to the next tokens in Line 0 until a match is found. If there's a match, mark the matched tokens in Line 1 so that it will not be matched again.
2. Return the number of matched tokens between Line 0 and Line 1.
3. Return the distance of the matched tokens. Example: token NN is found on position 3 on line 0 and position 5 on Line 1. Distance = |3-5| = 2

I've tried using split string and store it to String[] but String[] is fixed and doesn't allow shrinking or adding of new elements. Tried Pattern Matcher but with disasterous results. Tried a few other methods but there's some problems with my nested for loops..(will post part of my coding if it will help).

Any advice or pointers on how to solve this problem this would be very much appreciated. Thank you very much.

Answer 1

Think about the task in different ways. You want to scan for tokens (thus: Scanner), and you want to match the tokens (thus: a List, because you need order.) Then you'd iterate through the different collections for each token, noting the matches and the distance.

Answer 2

Have you tried using Scanner ?

If not, totally do. It would look like this:

String line1 = ... // your line 1
String line2 = ... // your line 2
Scanner s1 = new Scanner(line1); 

int i1 = 0;
while (s1.hasNext()) {
    String token1 = s1.next();
    Scanner s2 = new Scanner(line2);

    int i2 = 0;
    while (s2.hasNext()) {
        String token2 = s2.next();

        // now you have token1, token2 and their positions (i1, i2)
        // do whatever you want with them

        i2++;
    } // end reading line2
    i1++;
} // end reading line1

EDIT: Regarding your loops to select different lines in the Arraylist, what you need is to compare every array element to every other array element (which is probably the best thing to google if this explanation is lacking).

In Java that looks like this:

for (int i = 0; i < thearraylist.size()-1; i++) {
    for (int j = i+1; j < thearraylist.size(); j++) {

        // now the elements and indices i and j are compared
        // if we were running into my code above:

        String line1 = thearraylist.get(i);
        String line2 = thearraylist.get(j);

        // ... and then compare them

     }
}

The reason the second loop starts from i+1 is to eliminate these unnecessary comparisons:

1. Every element would be compared to itself at each point that j=i, which is useless. In the above loop, j starts at i+1 and increases, so it will never equal i.
2. Each pair of elements will be compared twice. For example, when i=0, j=1 you are comparing the first two elements. When i=1, j=0 you are also comparing the first two elements . This makes the second comparison redundant. To get rid of the second 'backwards' comparison, we insist that j always be higher than i.

If you find this confusing, I would highly recommend working it out on paper by listing the values of i and j as you move through the loop.