简体繁体 English

在hadoop / cascading中处理UTF-16LE编码的文件

[英]Process UTF-16LE encoded file in hadoop/cascading

原文 2018-02-08 09:49:28 6 1 java/ hadoop/ mapreduce/ cascading/ utf-16le

I need to process a UTF-16LE encoded file in cascading on top of hadoop. 我需要在hadoop上进行级联处理UTF-16LE编码的文件。 I've tried following approaches but none of these are working. 我尝试了以下方法，但是这些方法均无效。

While assigning value -Xmx1024m -Dfile.encoding=UTF-16LE to property mapreduce.map.java.opts in mapred-site.xml failed due to NullPointerException at: com.google.common.base.Preconditions.checkNotNull(Preconditions.java:187) But this method works for UTF-8. 在为mapred-site.xml中的属性mapreduce.map.java.opts分配值-Xmx1024m -Dfile.encoding=UTF-16LE ，由于NullPointerException而在com.google.common.base.Preconditions.checkNotNull(Preconditions.java:187)但是此方法适用于UTF-8。 Is hadoop unable to process UTF-16 data? hadoop无法处理UTF-16数据吗？
Doing System.setProperty("file.encoding", "UTF-16LE"); 正在做System.setProperty("file.encoding", "UTF-16LE"); in code is also unable to parse the data 在代码中也无法解析数据
Overriding charset of TextDelimited class of Cascading is also unable to process data 级联的TextDelimited类的重写字符集也无法处理数据

However using BufferedReader to read it in UTF-16LE parses the data correctly. 但是，使用BufferedReader在UTF-16LE中读取它可以正确解析数据。

Please help 请帮忙

Thanks in advance 提前致谢

1 个解决方案

在某处发现：Hadoop不支持UTF-16文件

如何使用 Java 处理 UTF-16LE 编码的文本文件？或将其转换为 ASCII？ - How to deal with UTF-16LE encoded text file using Java? or convert it to ASCII?

打开xls文件并使用Java和UTF-16LE将其另存为tsv文件到UTF-8转换 - opening xls file and saving it as tsv file using java and UTF-16LE to UTF-8 conversion

Java从UTF-16LE字符串解析XML - Java Parsing XML from UTF-16LE string

UTF-16LE编码和xerces2 Java - UTF-16LE encoding and xerces2 Java

使用UTF-16LE编码将字符串快速转换为byte [] - Fast conversion of String to byte[] using UTF-16LE encoding

困扰我教授的Java UTF-16LE古怪错误 - Java UTF-16LE Weird Error That Stumped My Professor

无法在 Android 版本 9 上使用编码 UTF-16LE - Unable to use encoding UTF-16LE on Android Version 9

使用 UTF-16LE 编码和 Apache Commons IO 读取和写入文本文件 - Reading and Writing Text files with UTF-16LE encoding and Apache Commons IO

如何使用BOM对UTF-16LE字节数组进行编码/解码？ - How do I encode/decode UTF-16LE byte arrays with a BOM?

如何在Java中验证和解析UTF-16编码的XML文件？ - How to validate and parse an UTF-16 encoded XML file in Java?

暂无

暂无

声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 如何使用 Java 处理 UTF-16LE 编码的文本文件？或将其转换为 ASCII？ - How to deal with UTF-16LE encoded text file using Java? or convert it to ASCII? 打开xls文件并使用Java和UTF-16LE将其另存为tsv文件到UTF-8转换 - opening xls file and saving it as tsv file using java and UTF-16LE to UTF-8 conversion Java从UTF-16LE字符串解析XML - Java Parsing XML from UTF-16LE string UTF-16LE编码和xerces2 Java - UTF-16LE encoding and xerces2 Java 使用UTF-16LE编码将字符串快速转换为byte [] - Fast conversion of String to byte[] using UTF-16LE encoding 困扰我教授的Java UTF-16LE古怪错误 - Java UTF-16LE Weird Error That Stumped My Professor 无法在 Android 版本 9 上使用编码 UTF-16LE - Unable to use encoding UTF-16LE on Android Version 9 使用 UTF-16LE 编码和 Apache Commons IO 读取和写入文本文件 - Reading and Writing Text files with UTF-16LE encoding and Apache Commons IO 如何使用BOM对UTF-16LE字节数组进行编码/解码？ - How do I encode/decode UTF-16LE byte arrays with a BOM? 如何在Java中验证和解析UTF-16编码的XML文件？ - How to validate and parse an UTF-16 encoded XML file in Java?

相关标签

粤ICP备18138465号 © 2020-2024 STACKOOM.COM