正则表达式要删除所有具有通用语言支持的非字母数字字符吗？

Question

I would like to use Pattern's compile method to do this. Such as

String text = "Where? What is that, an animal? No! It is a plane.";
Pattern p = new Pattern("*some regex here*");
String delim = p.matcher(text).replaceAll("");

What is the regex that can do what I'm trying to accomplish?

Example strings:

English

Input: "Where? What is that, an animal? No! It is a plane."
Output: "Where What is that an animal No It is a plane"

Spanish

Input: "¿Dónde? ¿Qué es eso, un animal? ¡No! Es un avión."
Output: "Dónde Qué es eso un animal No Es un avión"

Portuguese

Input: "Onde? O que é isso, um animal? Não! É um avião."
Output: "Onde O que é isso um animal Não É um avião"

Hopefully the examples make it clear what I'm trying to accomplish. Thanks all!

Answer 1

I am not an expert in all the languages of the world, however, your requirements could be met by doing this on a language specific basis:

Regex rgx = new Regex("[^a-zA-Z0-9 <put language specific characters to preserve here>]");
str = rgx.Replace(str, "");

I speak English and Korean, and can tell you that punctuation in Korean is identical to that used in English. As indicated above, you can add characters that should be preserved and not considered punctuation for a particular language. For example, let's say the tilde should not be considered punctuation. Then use the regex:

[^a-zA-Z0-9 ~]

Answer 2

The Java Pattern class, which is Java's implementation of regex, supports Unicode Categories , eg \\p{Lu} . Since you want alphanumeric, that would be Categories L (Letter) and N (Number).

Since your example shows you also want to keep spaces, you need to include that. Let's use the Predefined Character Class \\s , so you also get to keep newlines and tabs.

To find anything but the specified characters, use a Negation Character Class : [^abc]

All-in-all, that means [^\\s\\p{L}\\p{N}] :

String output = input.replaceAll("[^\\s\\p{L}\\p{N}]+", "");

Where What is that an animal No It is a plane
Dónde Qué es eso un animal No Es un avión
Onde O que é isso um animal Não É um avião

Or see regex101.com for demo.

---

Of course, there are multiple ways to do it.

You could alternatively use the POSIX Character Class \\p{Alnum} , and then enable UNICODE_CHARACTER_CLASS , using (?U) .

String output = input.replaceAll("(?U)[^\\s\\p{Alnum}]+", "");

Where What is that an animal No It is a plane
Dónde Qué es eso un animal No Es un avión
Onde O que é isso um animal Não É um avião

---

Now, if you didn't want spaces, that could be simplified by using \\P{xx} instead:

String output = input.replaceAll("(?U)\\P{Alnum}+", "");

WhereWhatisthatananimalNoItisaplane
DóndeQuéesesounanimalNoEsunavión
OndeOqueéissoumanimalNãoÉumavião