拆分字符串并保留字符（正则表达式模式）

Question

I would like to split a String and despair on the regex pattern.

I need to split a string like this: Hi I want "to split" this (String) to a String array like this:

String [] array = {"Hi", "I", "want", """, "to", "split", """, "this", "(", "string", ")"};

This is what I have tried, but it deletes the delimiter.

public static void main(String[] args) {

    String string = "Hi \"why should\" (this work)";

    String[] array;
    array = string.split("\\s"
            + "|\\s(?=\")"
            + "|\\w(?=\")"
            + "|\"(?=\\w)"
            + "|\\s(?=\\()"
            + "|\\w(?=\\))"
            + "|\\((?=\\w)");

    for (String str : array) {
        System.out.println(str);
    }
}

Result:

Hi

why
shoul
"

this
wor
)

Answer 1

You can match the tokens with the regex \\w+|[\\w\\s] , assuming that you want the punctuation characters to end up in different tokens:

String input = "Hi I want \"to split\" this (String).";

Matcher matcher = Pattern.compile("\\w+|[^\\w\\s]").matcher(input);
List<String> out = new ArrayList<>();

while (matcher.find()) {
    out.add(matcher.group());
}

The output ArrayList contains:

[Hi, I, want, ", to, split, ", this, (, String, ), .]

You might want to use (?U) flag to make the \\w and \\s follows the Unicode definition of word and whitespace character. By default, \\w and \\s only recognizes word and whitespace characters in ASCII range.

---

For the sake of completeness, here is the solution in split() , which works on Java 8 and above. There will be an extra empty string at the beginning in Java 7.

String tokens[] = input.split("\\s+|(?<![\\w\\s])(?=\\w)|(?<=\\w)(?![\\w\\s])|(?<=[^\\w\\s])(?=[^\\w\\s])");

The regex is rather complex, since the empty string splits between punctuation character and word character need to avoid the cases already split by \\s+ .

Since the regex in the split solution is quite a mess, please use the match solution instead .

Answer 2

What language are you trying to write this in?

You could write regex groups something like: (.+)(\\s)

This would match any quantity of characters followed by a space