java-从文本提取中省略特殊字符

Question

I have a program where I extract text or words from PDF file and insert those words to the table in database.

During insertion, I have a special regular expression to omit special characters if it exists on the word. The rule is if any words that has special characters in front of the word or at the end of the word, it gets removed.

Example:

Text : `,test.`
Token : `test`
Text: ?good
Token : good 
 Text: ?,.
 Token:
 Text: www.stack.com
 Token: www.stack.com

As long there is no space between characters, the special characters stay. This is at least how I defined it to be.

This is the general idea where I define my definition of what words to be stored. However, when it comes to certain words like underscore:

Text: _
Token : Same as above

Text: _—,m‘—_
Token : same as above

It doesn't seem to treat the underscore as a special character.

My code:

String lines[] = text.split("\\r?\\n");
    for (String line : lines) {
        String[] words = line.split(" ");

        System.out.println("Line: " + line);



        preparedStatement = con1.prepareStatement(sql);
        int i=0;
        for (String word : words) {

            // check if one or more special characters at end of string then remove OR
            // check special characters in beginning of the string then remove
            // insert every word directly to table db
            word = word.replaceAll("([\\W]+$)|(^[\\W]+)", "");
            preparedStatement.setString(1, path1);
            preparedStatement.setString(2, word);
              System.out.println("Token: " +word);
            preparedStatement.executeUpdate();
        }


    }

Is there a way to properly ignored every possible combination of special characters or symbols?

Answer 1

The definition of \\W is [^a-zA-Z_0-9] (see Java Pattern API ).

So to get the same behaviour without the underscores, replace \\W with [^a-zA-Z0-9]

Your line of code would then be:

word = word.replaceAll("([^a-zA-Z_0-9]+$)|(^[^a-zA-Z_0-9]+)", "");

Answer 2

您可以使用以下内容替换所有特殊字符（空格除外）。

word = word.replaceAll("([ a-zA-Z0-9])", "");