使用扫描仪和文本文件仅扫描单词

Question

I have to read in words from a file. For example a sentence might be

Bill's favorite animal is a dog. He is buying one at 1:30.

I need to only have the words, while not eliminating apostrophes, but eliminating the 1:30 . The desired out put for this would start:

• Bill's
• favorite
• animal,
• ...etc.

Code:

Scanner scanner = null;
Pattern pattern=Pattern.compile("[^\\w+]");
String word;

try{
    scanner=new Scanner(file);
}catch(FileNotFoundException e){
    System.out.println("Can't Find the File in Dictionary class!");
}
time=System.nanoTime();
while(scanner.hasNext()){
    scanner.useDelimiter(pattern);
    word=scanner.next();
    System.out.println(word);
    if(!word.equals("")){
        dictionary.add(word);
    }
}

I have tried using delimiter, but that results in Bill and s on separate lines with no ' . I was hoping simply to be able to use

scanner.next(Pattern.compile("[^\\w+]));

but when I try that I get an InputMismatchException. Hopefully someone can help with this! Thanks!

Answer 1

The pattern "[^\\\\w+]" is wrong. It is matching any character which is not a letter, digit, underscore, or a plus sign. The plus sign here is not a quantifier, so if your sample text contained "Bill got an A+" it would find the words "Bill", "got", "an", and "A+". Is this what you want? It seems more likely you meant to write "[^\\\\w]+" , which would eliminate the empty strings from the results when there is a run of delimiter characters.

It seems like you can just add the apostrophe to the pattern. If we also move the stray plus sign, that results in a pattern of "[^\\\\w']+" , however, while this is closer, it still includes digits so you will get "1" and "30" as words from "1:30".

I think what you really want is "[^\\\\p{Alpha}']+" which will use runs of one or more characters that are not letters or apostrophes as delimiters, and thus match all runs of letters and apostrophes as tokens. The output would be the following tokens:

• Bill's
• favorite
• animal
• is
• a
• dog
• He
• is
• buying
• one
• at