[英]How to remove accents from a unicode string in java using a hashmap?
I am using the following link to create a hashmap of key = unicode value of characters and value being the actual character it should map to - https://github.com/lmjabreu/solr-conftemplate/blob/master/mapping-ISOLatin1Accent.txt 我正在使用以下链接来创建键= Unicode字符值的哈希图,值是应映射到的实际字符-https: //github.com/lmjabreu/solr-conftemplate/blob/master/mapping-ISOLatin1Accent。文本
So far I have written the following code to remove accents from the string 到目前为止,我已经编写了以下代码来删除字符串中的重音符号
public class ACCENTS {
public static void main(String[] args){
// this is the hashmap that stores the mappings of the characters to their ascii equivalent
HashMap<Character, Character> characterMappings = new HashMap<>();
characterMappings.put('\u00C0', 'A');
characterMappings.put('\u00C1', 'A');
characterMappings.put('\u00C2', 'A');
characterMappings.put('\u00C3', 'A');
characterMappings.put('\u00C4', 'A');
characterMappings.put('\u00C5', 'A');
characterMappings.put('\u00C7','C');
characterMappings.put('\u00C8', 'E');
characterMappings.put('\u00C9','E');
characterMappings.put('\u00CA', 'E');
characterMappings.put('\u00CB', 'E');
characterMappings.put('\u00CC', 'I');
characterMappings.put('\u00CD', 'I');
characterMappings.put('\u00CE', 'I');
characterMappings.put('\u00CF', 'I');
characterMappings.put('\u00D0', 'D');
characterMappings.put('\u00D1', 'N');
characterMappings.put('\u00D2', 'O');
characterMappings.put('\u00D3', 'O');
characterMappings.put('\u00D4', 'O');
characterMappings.put('\u00D5', 'O');
characterMappings.put('\u00D6', 'O');
characterMappings.put('\u00D8', 'O');
characterMappings.put('\u00D9', 'U');
characterMappings.put('\u00DA', 'U');
characterMappings.put('\u00DB', 'U');
characterMappings.put('\u00DC', 'U');
characterMappings.put('\u00DD', 'Y');
characterMappings.put('\u0178', 'Y');
characterMappings.put('\u00E0', 'a');
characterMappings.put('\u00E1', 'a');
characterMappings.put('\u00E2', 'a');
characterMappings.put('\u00E3','a');
characterMappings.put('\u00E4', 'a');
characterMappings.put('\u00E5', 'a');
characterMappings.put('\u00E7', 'c');
characterMappings.put('\u00E8', 'e');
characterMappings.put('\u00E9', 'e');
characterMappings.put('\u00EA','e');
characterMappings.put('\u00EB', 'e');
characterMappings.put('\u00EC', 'i');
characterMappings.put('\u00ED', 'i');
characterMappings.put('\u00EE', 'i');
characterMappings.put('\u00EF', 'i');
characterMappings.put('\u00F0', 'd');
characterMappings.put('\u00F1','n' );
characterMappings.put('\u00F2', 'o');
characterMappings.put('\u00F3', 'o');
characterMappings.put('\u00F4', 'o');
characterMappings.put('\u00F5', 'o');
characterMappings.put('\u00F6', 'o');
characterMappings.put('\u00F8', 'o');
characterMappings.put('\u00F9', 'u');
characterMappings.put('\u00FA', 'u');
characterMappings.put('\u00FB', 'u');
characterMappings.put('\u00FC', 'u');
characterMappings.put('\u00FD', 'y');
characterMappings.put('\u00FF', 'y');
String token = "nа̀ра";
String newString = "";
for(int i = 0 ; i < token.length() ; ++i){
if( characterMappings.containsKey(token.charAt(i)) )
newString += characterMappings.get(token.charAt(i));
else
newString += token.charAt(i);
}
System.out.println(newString);
}
}
The expected result should have been "napa" but it turns out no conversion is being performed, what can be a possible cause of deviation for this case, I am not able to find one. 预期结果应该是“ napa”,但事实证明没有执行任何转换,这可能是导致这种情况偏离的原因,我找不到。
Not shure why you want to use a HashMap. 不知道为什么要使用HashMap。 But if you just want to remove the diacritics perhaps this helps:
但是,如果您只想删除变音符号,则可能会有所帮助:
String s = "nа̀ра";
s = Normalizer.normalize( s, Normalizer.Form.NFD );
s = s.replaceAll("\\p{InCombiningDiacriticalMarks}+", "");
System.out.println( s );
--> napa ->纳帕
(If you insist on using the HashMap you should have still a look a the 'Normalizer' class because it can work in the other direction, too.) (如果您坚持使用HashMap,则应该再看看'Normalizer'类,因为它也可以在另一个方向上工作。)
Taken from this article: http://blog.smartkey.co.uk/2009/10/how-to-strip-accents-from-strings-using-java-6/ 摘自本文: http : //blog.smartkey.co.uk/2009/10/how-to-strip-accents-from-strings-using-java-6/
you ran into some of the ugliest 'features' of Java: One unicode character may be represented by a tupel (and even a tripel) of characters. 您遇到了Java的一些最丑陋的“特征”:一个unicode字符可能由一个字符的tupel(甚至是tripel)来表示。
In fact, token has a length of 5 chars. 实际上,令牌的长度为5个字符。 á is a combination of two chars and can only be represented as a String.
á是两个字符的组合,只能表示为字符串。
This is why 这就是为什么
characterMappings.put('а̀`', 'y'); //(accent can't be displayed correctly in code-mode, try it yourself)
won't compile. 不会编译。
Here is a more explaination. 这是更多说明。
In my humble oppinion String is one of the worst classes in Java. 在我看来,String是Java中最差的类之一。 Especially if you use 'non standard' characters.
特别是如果您使用“非标准”字符。
To solve your problem I would suggest changing your map to Map<String,String>
or Map<String,Character>
. 为了解决您的问题,我建议将地图更改为
Map<String,String>
或Map<String,Character>
。 This way you can map your 'characters' and as a neat sideeffect your code becomes more readable if you dismiss the escaped unicode-characters. 这样,您就可以映射“字符”,并且,如果您消除了转义的unicode字符,那么整洁的副作用将使您的代码更具可读性。
For more information google for HighSurrogate or CodePoint. 有关更多信息,请谷歌查询HighSurrogate或CodePoint。 CodePoints are valid (=displayable) char-sequences, which - as mentioned before - need not to necessarily correspond with the number of chars in a String.
代码点是有效的(=可显示的)字符序列,如前所述,这些字符序列不必一定与字符串中的字符数相对应。
This is necessary because a Java-Character is just 2 byte wide. 这是必需的,因为Java字符只有2个字节宽。 To small for all unicode characters, but big enough most of the time (=as long as you use standard latin characters).
对于所有unicode字符,请减小为该大小,但在大多数情况下要足够大(=,只要您使用标准的拉丁字符即可)。
Edit: 编辑:
Even with a Map<String,String>
, your code won't work, cause you still loop over chars. 即使使用
Map<String,String>
,您的代码也无法正常工作,因为您仍然会遍历char。 But no single Java-character will match you special unicode-character. 但是没有一个Java字符会与您的特殊unicode字符匹配。
This might help, though it may not work under any circumstances (java strings are nasty after all): 这可能会有所帮助,尽管在任何情况下都可能不起作用(毕竟Java字符串很讨厌):
HashMap<String, String> characterMappings = new HashMap<>();
characterMappings.put("а̀", "a");
String token = "nа̀ра";
String newString = "";
for (Entry<String, String> e : characterMappings.entrySet()) {
token = token.replaceAll(e.getKey(), e.getValue());
}
System.out.println(token);
Edit 2 编辑2
Since posting code as a comment sucks: 由于将代码作为注释发布很烂:
String s = "brûlée";
String s1 = Normalizer.normalize(s, Normalizer.Form.NFKD);
String regex = "[\\p{InCombiningDiacriticalMarks}\\p{IsLm}\\p{IsSk}]+";
String s2 = new String(s1.replaceAll(regex, "").getBytes("ascii"),
"ascii");
System.out.println(s2);
this works for me with everything I tried so far. 到目前为止,我所做的一切都对我有用。 Still @Scheintod deserves the credit.
@Scheintod仍然值得赞扬。 Source found here
在这里找到源
Best regards 最好的祝福
sam SAM
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.