[英]Force flush on a GZIPOutputStream in java
我们正在开发一个需要刷新(强制压缩和发送数据)GZIPOutputStream 的程序。 问题是, GZIPOutputStream 的刷新方法没有按预期工作(强制压缩并发送数据),而是 Stream 等待更多数据以进行有效的数据压缩。
当您调用完成时,数据被压缩并通过输出流发送,但 GZIPOutputStream(不是底层流)将关闭,因此我们无法写入更多数据,直到我们创建新的 GZIPOutputStream,这会消耗时间和性能。
希望任何人都可以帮助解决这个问题。
最好的问候。
我还没有尝试过这个,这个建议在我们手头有 Java 7 之前不会有用,但是从DeflaterOutputStream
继承的GZIPOutputStream
的flush()
方法的文档依赖于在构建时使用syncFlush
指定的刷新模式参数(与Deflater#SYNC_FLUSH
相关)来决定是否刷新待压缩的待处理数据。 这个syncFlush
参数在构建时也被GZIPOutputStream
接受。
听起来您想使用Deflator#SYNC_FLUSH
甚至Deflater#FULL_FLUSH
,但是,在深入研究之前,首先尝试使用两个参数或四个参数的GZIPOutputStream
构造函数,并为syncFlush
参数传递true
。 这将激活您想要的冲洗行为。
我没有找到其他工作答案。 它仍然拒绝刷新,因为 GZIPOutputStream 使用的本机代码保留了数据。
幸运的是,我发现有人实现了 FlushableGZIPOutputStream 作为 Apache Tomcat 项目的一部分。 这是神奇的部分:
@Override
public synchronized void flush() throws IOException {
if (hasLastByte) {
// - do not allow the gzip header to be flushed on its own
// - do not do anything if there is no data to send
// trick the deflater to flush
/**
* Now this is tricky: We force the Deflater to flush its data by
* switching compression level. As yet, a perplexingly simple workaround
* for
* http://developer.java.sun.com/developer/bugParade/bugs/4255743.html
*/
if (!def.finished()) {
def.setLevel(Deflater.NO_COMPRESSION);
flushLastByte();
flagReenableCompression = true;
}
}
out.flush();
}
你可以在这个 jar 中找到整个类(如果你使用 Maven):
<dependency>
<groupId>org.apache.tomcat</groupId>
<artifactId>tomcat-coyote</artifactId>
<version>7.0.8</version>
</dependency>
或者只是去获取源代码FlushableGZIPOutputStream.java
它是在 Apache-2.0 许可下发布的。
这段代码在我的应用程序中对我很有用。
public class StreamingGZIPOutputStream extends GZIPOutputStream {
public StreamingGZIPOutputStream(OutputStream out) throws IOException {
super(out);
}
@Override
protected void deflate() throws IOException {
// SYNC_FLUSH is the key here, because it causes writing to the output
// stream in a streaming manner instead of waiting until the entire
// contents of the response are known. for a large 1 MB json example
// this took the size from around 48k to around 50k, so the benefits
// of sending data to the client sooner seem to far outweigh the
// added data sent due to less efficient compression
int len = def.deflate(buf, 0, buf.length, Deflater.SYNC_FLUSH);
if (len > 0) {
out.write(buf, 0, len);
}
}
}
Android
也有同样的问题。 接受者的回答不起作用,因为def.setLevel(Deflater.NO_COMPRESSION);
抛出异常。 根据flush
方法,它改变了Deflater
压缩级别。 所以我认为应该在写入数据之前调用更改压缩,但我不确定。
还有 2 个其他选项:
错误 ID 4813885处理此问题。 2006 年 9 月 9 日提交的“DamonHD”评论(大约是错误报告的一半)包含一个FlushableGZIPOutputStream
示例,该示例构建在Jazzlib 的net.sf.jazzlib.DeflaterOutputStream
之上。
作为参考,这里有一个(重新格式化的)摘录:
/**
* Substitute for GZIPOutputStream that maximises compression and has a usable
* flush(). This is also more careful about its output writes for efficiency,
* and indeed buffers them to minimise the number of write()s downstream which
* is especially useful where each write() has a cost such as an OS call, a disc
* write, or a network packet.
*/
public class FlushableGZIPOutputStream extends net.sf.jazzlib.DeflaterOutputStream {
private final CRC32 crc = new CRC32();
private final static int GZIP_MAGIC = 0x8b1f;
private final OutputStream os;
/** Set when input has arrived and not yet been compressed and flushed downstream. */
private boolean somethingWritten;
public FlushableGZIPOutputStream(final OutputStream os) throws IOException {
this(os, 8192);
}
public FlushableGZIPOutputStream(final OutputStream os, final int bufsize) throws IOException {
super(new FilterOutputStream(new BufferedOutputStream(os, bufsize)) {
/** Suppress inappropriate/inefficient flush()es by DeflaterOutputStream. */
@Override
public void flush() {
}
}, new net.sf.jazzlib.Deflater(net.sf.jazzlib.Deflater.BEST_COMPRESSION, true));
this.os = os;
writeHeader();
crc.reset();
}
public synchronized void write(byte[] buf, int off, int len) throws IOException {
somethingWritten = true;
super.write(buf, off, len);
crc.update(buf, off, len);
}
/**
* Flush any accumulated input downstream in compressed form. We overcome
* some bugs/misfeatures here so that:
* <ul>
* <li>We won't allow the GZIP header to be flushed on its own without real compressed
* data in the same write downstream.
* <li>We ensure that any accumulated uncompressed data really is forced through the
* compressor.
* <li>We prevent spurious empty compressed blocks being produced from successive
* flush()es with no intervening new data.
* </ul>
*/
@Override
public synchronized void flush() throws IOException {
if (!somethingWritten) { return; }
// We call this to get def.flush() called,
// but suppress the (usually premature) out.flush() called internally.
super.flush();
// Since super.flush() seems to fail to reliably force output,
// possibly due to over-cautious def.needsInput() guard following def.flush(),
// we try to force the issue here by bypassing the guard.
int len;
while((len = def.deflate(buf, 0, buf.length)) > 0) {
out.write(buf, 0, len);
}
// Really flush the stream below us...
os.flush();
// Further flush()es ignored until more input data data written.
somethingWritten = false;
}
public synchronized void close() throws IOException {
if (!def.finished()) {
def.finish();
do {
int len = def.deflate(buf, 0, buf.length);
if (len <= 0) {
break;
}
out.write(buf, 0, len);
} while (!def.finished());
}
// Write trailer
out.write(generateTrailer());
out.close();
}
// ...
}
你可能会发现它很有用。
正如@seh 所说,这很好用:
ByteArrayOutputStream stream = new ByteArrayOutputStream();
// the second param need to be true
GZIPOutputStream gzip = new GZIPOutputStream(stream, true);
gzip.write( .. );
gzip.flush();
...
gzip.close()
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.