Storm的Java UTF8编码问题

Question

Pretty desperate for help after 2 days trying to debug this issue.

I have some text that contains unicode characters, for example, the word:

korte støvler

If I run code that writes this word to a file on one of the problem machines, it works correctly. However, when I write the file exactly the same way in a storm bolt, it does not encode correctly, and the ø character is replaced with question marks.

In the storm_env.ini file I have set

STORM_JAR_JVM_OPTS:-Dfile.encoding=UTF-8

I also set the encoding as UTF-8 in the code, and in mvn when it is packaged.

I have run tests on the boxes to check JVM default encodings, and they are all UTF-8.

I have tried 3 different methods of writing the file and all cause the same issue, so it is definitely not that.

Answer 1

This issue was fixed by simply build another machine on ec2. It had exactly the same software versions and configuration as the boxes with issues.