[英]Possible memory leak when caching BufferedImage
We have an application which serve images, to speed up the response time, we cache the BufferedImage
directly in memory. 我们有一个提供图像的应用程序,为了加快响应时间,我们将BufferedImage
直接缓存在内存中。
class Provider {
@Override
public IData render(String... layers,String coordinate) {
int rwidth = 256 , rheight = 256 ;
ArrayList<BufferedImage> result = new ArrayList<BufferedImage>();
for (String layer : layers) {
String lkey = layer + "-" + coordinate;
BufferedImage imageData = cacher.get(lkey);
if (imageData == null) {
try {
imageData = generateImage(layer, coordinate,rwidth, rheight, bbox);
cacher.put(lkey, imageData);
} catch (IOException e) {
e.printStackTrace();
continue;
}
}
if (imageData != null) {
result.add(imageData);
}
}
return new Data(rheight, rheight, width, result);
}
private BufferedImage generateImage(String layer, String coordinate,int rwidth, int rheight) throws IOException {
BufferedImage image = new BufferedImage(rwidth, rheight, BufferedImage.TYPE_INT_ARGB);
Graphics2D g = image.createGraphics();
g.setColor(Color.RED);
g.drawString(layer+"-"+coordinate, new Random().nextInt(rwidth), new Random().nextInt(rheight));
g.dispose();
return image;
}
}
class Data implements IData {
public Data(int imageWidth, int imageHeight, int originalWidth, ArrayList<BufferedImage> images) {
this.imageResult = new BufferedImage(this.imageWidth, this.imageHeight, BufferedImage.TYPE_INT_ARGB);
Graphics2D g = imageResult.createGraphics();
for (BufferedImage imgData : images) {
g.drawImage(imgData, 0, 0, null);
imgData = null;
}
imageResult.flush();
g.dispose();
images.clear();
}
@Override
public void save(OutputStream out, String format) throws IOException {
ImageIO.write(this.imageResult, format, out);
out.flush();
this.imageResult = null;
}
}
usage: 用法:
class ImageServlet extends HttpServlet {
void doGet(req,res){
IData data= provider.render(req.getParameter("layers").split(","));
OutputStream out=res.getOutputStream();
data.save(out,"png")
out.flush();
}
}
Note:the provider
filed is a single instance. 注意: provider
提交的是单个实例。
However it seems that there is a possible memory leak because I will get Out Of Memory
exception when the application keep running for about 2 minutes. 但是,似乎存在可能的内存泄漏,因为当应用程序继续运行大约2分钟时,我将获得Out Of Memory
异常。
Then I use visualvm
to check the memory usage: 然后我使用visualvm
来检查内存使用情况:
Even I Perform GC
manually, the memory can not be released. 即使我手动Perform GC
,也无法释放内存。
And Though there are only 300+ BufferedImage
cached, and 20M+
memory are used, 1.3G+
memory are retained. 虽然只缓存了300多个BufferedImage
,并且使用了20M+
内存,但仍保留了1.3G+
内存。 In fact, through "firebug" I can make sure that a generate image is less than 1Kb
. 事实上,通过“firebug”,我可以确保生成的图像小于1Kb
。 So I think the memory usage is not healthy. 所以我认为内存使用不健康。
Once I do not use the cache (comment the following line): 一旦我不使用缓存(注释以下行):
//cacher.put(lkey, imageData);
The memory usage looks good: 内存使用率看起来很好:
So it seem that the cached BufferedImage
cause the memory leak. 所以看起来缓存的BufferedImage
会导致内存泄漏。
Then I tried to transform the BufferedImage
to byte[]
and cache the byte[]
instead of the object itself. 然后我尝试将BufferedImage
转换为byte[]
并缓存byte[]
而不是对象本身。 And the memory usage is still normal. 内存使用情况仍然正常。 However I found the Serialization
and Deserialization
for the BufferedImage
will cost too much time. 但是我发现BufferedImage
的Serialization
和Deserialization
Serialization
Deserialization
将花费太多时间。
So I wonder if you guys have any experience of image caching? 所以我想知道你们有没有图像缓存的经验?
update: 更新:
Since there are so many people said that there is no memory leak but my cacher use too many memory, I am not sure but I have tried to cache byte[]
instead of BufferedImage
directly, and the memory use looks good. 既然有这么多人说没有内存泄漏,但是我的cacher使用了太多的内存,我不确定,但我试图直接缓存byte[]
而不是BufferedImage
,内存使用看起来不错。 And I can not imagine 322 image will take up 1.5G+ memory,event as @BrettOkken said, the total size should be (256 * 256 * 4byte) * 322 / 1024 / 1024 = 80M
, far less than 1Gb. 而我无法想象322图像会占用1.5G +内存,事件如@BrettOkken所说,总大小应为(256 * 256 * 4byte) * 322 / 1024 / 1024 = 80M
322/1024/1024 (256 * 256 * 4byte) * 322 / 1024 / 1024 = 80M
,远小于1Gb。
And just now,I change to cache the byte
and monitor the memory again, codes change like this: 刚才,我改为缓存byte
并再次监视内存,代码改变如下:
BufferedImage ig = generateImage(layer,coordinate rwidth, rheight);
ByteArrayOutputStream bos = new ByteArrayOutputStream();
ImageIO.write(ig, "png", bos);
imageData = bos.toByteArray();
tileCacher.put(lkey, imageData);
And the memory usage: 和内存使用情况:
Same codes, same operation. 相同的代码,相同的操作。
Note from both VisualVM screenshots that 97.5% memory consumed by 4,313 instances of int[] (Which I assume is by cached buffered image) is not consumed in non-cached version. 从两个VisualVM屏幕截图中注意到,4,313个int []实例消耗了97.5%的内存(我假设是通过高速缓存的缓存图像),非高速缓存版本不会消耗这些内存。
Although you have a less than 1K PNG image (which is compressed as per PNG format), this single image is being generated out of multiple instances of buffered image (which is not compressed). 虽然您有一个小于1K的PNG图像(根据PNG格式压缩),但这个单个图像是由多个缓冲图像实例(未压缩)生成的。 Hence you cannot directly co-relate image size from browser to memory occupied on server. 因此,您无法直接将浏览器的图像大小与服务器上占用的内存联系起来。 So issue here is not memory leak but amount of memory required to cache this uncompressed layers of buffered images. 所以这里的问题不是内存泄漏,而是缓存这些未压缩的缓冲图像层所需的内存量。
Strategy to resolve this is to tweak your caching mechanism: 解决此问题的策略是调整您的缓存机制:
Not sure what caching API you are using or what are actual values in your request. 不确定您使用的是哪种缓存API,或者您的请求中的实际值是什么。 However based of visualvm it looks to me that String objects are leaking. 但是基于visualvm,我认为String对象正在泄漏。 Also as you mentioned if you turn off caching, problem is resolved. 正如您所提到的,如果您关闭缓存,问题就解决了。
Consider extract of below snippet of your code. 考虑下面代码片段的摘录。
String lkey = layer + "-" + coordinate;
BufferedImage imageData = cacher.get(lkey);
Now here are few things for you to consider for this code. 现在,您可以在此处为此代码考虑一些事项。
VisualVM is a start but it doesn't give the complete picture. VisualVM是一个开始,但它没有提供完整的图片。
You need to trigger a heap dump while the application is using a high amount of memory. 您需要在应用程序使用大量内存时触发堆转储。 You can trigger a heap dump from VisualVM. 您可以从VisualVM触发堆转储。 It can also be done automatically on an OOME if you add this vmarg to the java process: 如果将此vmarg添加到java进程,它也可以在OOME上自动完成:
-XX:+HeapDumpOnOutOfMemoryError
Use Memory Analyzer Tool to open and inspect the heap dump. 使用Memory Analyzer Tool打开并检查堆转储。
The tool is quite capable and can help you walk the object references to discover: 该工具非常强大,可以帮助您遍历对象引用以发现:
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.