[英]Java's GZIPOutputStream refuses to flush
Here is a JUnit test that demonstrates my issue:这是演示我的问题的 JUnit 测试:
package stream;
import static org.junit.jupiter.api.Assertions.*;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.OutputStreamWriter;
import java.io.PrintWriter;
import java.nio.ByteBuffer;
import java.util.Arrays;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.zip.GZIPInputStream;
import java.util.zip.GZIPOutputStream;
import org.junit.jupiter.api.Test;
class StreamTest {
public static class LoopbackStream {
private final byte[] END_MARKER = new byte[0];
private final ArrayBlockingQueue<byte[]> queue = new ArrayBlockingQueue<>(1024);
public OutputStream getOutputStream() {
return new OutputStream() {
@Override
public void write(int b) throws IOException {
this.write(new byte[] { (byte) b });
}
@Override
public void write(byte[] b, int off, int len) {
try {
queue.put(Arrays.copyOfRange(b, off, len - off));
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
@Override
public void close() {
try {
queue.put(END_MARKER);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
};
}
public InputStream getInputStream() {
return new InputStream() {
private boolean finished = false;
private ByteBuffer current = ByteBuffer.wrap(new byte[0]);
@Override
public int read() {
if (ensureData()) {
return Byte.toUnsignedInt(current.get());
} else {
return -1;
}
}
@Override
public int read(byte[] b, int off, int len) {
if (ensureData()) {
int position = current.position();
current.get(b, off, Math.min(len, current.remaining()));
return current.position() - position;
} else {
return -1;
}
}
private boolean ensureData() {
if (!finished && !current.hasRemaining()) {
try {
byte[] data = queue.take();
current = ByteBuffer.wrap(data);
finished = data == END_MARKER;
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
return false;
}
}
return !finished;
}
};
}
}
@Test
void testVanilla() throws IOException {
LoopbackStream objectUnderTest = new LoopbackStream();
PrintWriter pw = new PrintWriter(new OutputStreamWriter(objectUnderTest.getOutputStream()), true);
BufferedReader br = new BufferedReader(new InputStreamReader(objectUnderTest.getInputStream()));
pw.println("Hello World!");
assertEquals("Hello World!", br.readLine());
}
@Test
void testVanilla2() throws IOException {
LoopbackStream objectUnderTest = new LoopbackStream();
PrintWriter pw = new PrintWriter(new OutputStreamWriter(objectUnderTest.getOutputStream()), true);
BufferedReader br = new BufferedReader(new InputStreamReader(objectUnderTest.getInputStream()));
pw.println("Hello World!");
assertEquals("Hello World!", br.readLine());
pw.println("Hello Otherworld!");
assertEquals("Hello Otherworld!", br.readLine());
}
@Test
void testGzipped() throws IOException {
LoopbackStream objectUnderTest = new LoopbackStream();
PrintWriter pw = new PrintWriter(new OutputStreamWriter(new GZIPOutputStream(objectUnderTest.getOutputStream(), true)), true);
BufferedReader br = new BufferedReader(new InputStreamReader(new GZIPInputStream(objectUnderTest.getInputStream())));
pw.println("Hello World!");
assertEquals("Hello World!", br.readLine());
}
}
There are two individual tests.有两个单独的测试。 One that uses vanilla input and output streams (which works fine) and another that wraps those streams in their gzip equivalents.
一个使用 vanilla 输入和 output 流(工作正常),另一个将这些流包装在它们的 gzip 等效项中。
I've used the GZIPOutputStream
's syncFlush
option which I am expecting to automatically flush any remaining bytes from the stream whenever the parent stream is flushed.我已经使用了
GZIPOutputStream
的syncFlush
选项,我希望在刷新父 ZF7B44CFFAD5C52223D5498196C8A2E7BZ 时自动刷新 stream 中的任何剩余字节。 I'm using the PrintWriter
's autoFlush
option to flush its data whenever it does a println
.每当它执行
println
时,我都会使用PrintWriter
的autoFlush
选项来刷新其数据。
Is there a better way to force the GZIPOutputStream
to flush its buffers after a println
?有没有更好的方法来强制
GZIPOutputStream
在println
之后刷新其缓冲区?
I know that this is not the full answer to your question, but it is too long for a comment...我知道这不是您问题的完整答案,但是评论太长了...
Update:更新:
After further investigation it seems that it's not the GZIPOutputStream
that doesn't flush (by adding System.out.println("xy");
statements in the public void write(byte[] b, int off, int len)
method you can see that the GZIPOutputStream
writes two byte arrays into your OutputStream
: one is the gzip stream header, the other one is the encoded content of the first line of text).经过进一步调查,似乎不是
GZIPOutputStream
不刷新(通过在public void write(byte[] b, int off, int len)
方法中添加System.out.println("xy");
语句,您可以看到GZIPOutputStream
将两个字节 arrays 写入您的OutputStream
:一个是 gzip stream header,另一个是编码内容的第一行)。
It seems that the reading process blocks because of a bad interaction between the java.io.InputStreamReader
(respectively the sun.nio.cs.StreamDecoder
it uses) and the GZIPInputStream
.似乎由于
java.io.InputStreamReader
(分别是它使用的sun.nio.cs.StreamDecoder
)和GZIPInputStream
之间的不良交互而导致读取过程阻塞。
Basically, if the StreamDecoder
needs to read bytes from the underlying stream it tries to read as many bytes as possible (as long as underlying stream reports in.available() > 0
, implying that the underlying stream can yield some more bytes without blocking) Basically, if the
StreamDecoder
needs to read bytes from the underlying stream it tries to read as many bytes as possible (as long as underlying stream reports in.available() > 0
, implying that the underlying stream can yield some more bytes without blocking)
The problem with this check is that the InflaterInputStream
(the superclass of the GZIPInputStream
) always returns 1
for the number of available bytes, even if its source stream has no bytes available ( see the source of InflaterInputStream.available()
)这个检查的问题是
InflaterInputStream
( GZIPInputStream
的超类)总是返回1
的可用字节数,即使它的源 stream没有可用的字节( 参见 InflaterInputStream.available InflaterInputStream.available()
的源)
So it seems that while you can write line by line into a GZIPOutputStream
, you cannot easily read line by line from a GZIPInputStream
...所以看起来虽然你可以逐行写入
GZIPOutputStream
,但你不能轻易地从GZIPInputStream
中逐行读取......
Original answer:原答案:
The problem is not the GZIPOutputStream
, the problem is with the boolean ensureData()
method that refuses to read more than one block.问题不在于
GZIPOutputStream
,问题在于拒绝读取多个块的boolean ensureData()
方法。
The following test fails with vanilla streams too:以下测试也因香草流而失败:
@Test
void testVanilla2() throws IOException {
LoopbackStream objectUnderTest = new LoopbackStream();
PrintWriter pw = new PrintWriter(new OutputStreamWriter(objectUnderTest.getOutputStream()), true);
BufferedReader br = new BufferedReader(new InputStreamReader(objectUnderTest.getInputStream()));
pw.println("Hello World!");
assertEquals("Hello World!", br.readLine());
pw.println("Hello Otherworld!");
assertEquals("Hello Otherworld!", br.readLine());
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.