[英]Gson.toJson throws NullPointerException when the file size > 1GB
I tried to write into Json format in Java, but encountered NullPointerException when the file size is >1GB. 我试图用Java写入Json格式,但是当文件大小> 1GB时遇到NullPointerException。 Can anyone helps me to fix this issue?
任何人都可以帮助我解决这个问题吗?
The code keeps generating Json files, and the size of the files keep increasing. 代码不断生成Json文件,文件大小不断增加。 Once the file size > 1GB, the code throws exception as shown below.
一旦文件大小> 1GB,代码抛出异常,如下所示。 I used different data set for testing, so I don't think it is the data issue.
我使用不同的数据集进行测试,所以我不认为这是数据问题。 My guess is that there is a size limit for Gson.toJson in Java.
我的猜测是Java中的Gson.toJson有一个大小限制。
My code is: 我的代码是:
private HashMap<String,HashSet<Token>> tokenCounter = new HashMap<String,HashSet<Token>>();
....
private void writeToFile(){
try {
PrintWriter out = new PrintWriter(outputFileName);
out.println(gson.toJson(tokenCounter));
out.close();
} catch (IOException e) {
e.printStackTrace();
}
}
The exception it throws is: 抛出的例外是:
java.lang.NullPointerException
at java.lang.String.<init>(String.java:301)
at java.lang.StringBuffer.toString(StringBuffer.java:790)
at java.io.StringWriter.toString(StringWriter.java:204)
at com.google.gson.Gson.toJson(Gson.java:481)
at com.google.gson.Gson.toJson(Gson.java:460)
at com.ebay.classification.discovery.DailyDiscovery.writeToFile(DailyDiscovery.java:181)
at com.ebay.classification.discovery.DailyDiscovery.run(DailyDiscovery.java:169)
at com.ebay.classification.discovery.TestDailyDiscoveryContinue.run(TestDailyDiscoveryContinue.java:142)
at com.ebay.classification.discovery.TestDailyDiscoveryContinue.main(TestDailyDiscoveryContinue.java:245)
Posted as an answer to get around formatting issues in comments. 发布为答案,以解决评论中的格式问题。
An array of 2^30 char would be 2^31 bytes. 2 ^ 30个字符的数组将是2 ^ 31个字节。 As a single string, this is huge!
作为单个字符串,这是巨大的! The obvious question that needs to be asked is why you have the code:
需要提出的一个显而易见的问题是为什么你有代码:
PrintWriter out = new PrintWriter(outputFileName);
out.println(gson.toJson(tokenCounter));
out.close();
This can easily be written as: 这很容易写成:
FileWriter out = new FileWriter(outputFileName);
gson.toJson(tokenCounter, out);
out.flush();
out.close();
This would have no significant memory impact, and would be much faster. 这不会对内存造成重大影响,而且速度会快得多。
This does not answer the question why you get the NPE in a large StringWriter, but, frankly, what you are doing is absurd.... 这并没有回答为什么你在大型StringWriter中获得NPE的问题,但是,坦率地说,你所做的是荒谬的....
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.