[英]Proxy around a Writable stream cause `write after end` errors when piping data
I want to create a wrapper around a Writable stream.我想围绕可写流创建一个包装器。 But when I try to
pipe()
some data through that proxy, I end up having write after end errors.但是当我尝试通过该代理
pipe()
一些数据时,我最终会在结束错误后写入。
Here is a self-contained example showing my problem:这是一个显示我的问题的独立示例:
const http = require("http");
const fs = require("fs");
const { Writable, Readable } = require('stream');
class MyStream extends Readable {
constructor() {
super();
this._iterator = this.generator();
}
*generator() {
for(const c of "Hello")
yield c;
}
_read(n) {
const it = this._iterator.next();
if (it.done)
this.push(null);
else
this.push(it.value);
}
}
class Proxy extends Writable {
constructor(req) {
super();
this._node_req = req;
}
end(chunk, encoding, cb) {
console.trace("end");
this._node_req.end(chunk, encoding, cb);
return {}; // return something
}
_write(chunk, encoding, cb) {
console.log("_write", chunk.toString("utf8"));
return this._node_req.write(chunk, encoding, cb);
}
}
const src = new MyStream()
const req = http.request("http://httpbingo.org/post", { method: "POST" }, (res) => {
res.pipe(process.stderr);
});
src.pipe(new Proxy(req));
Here is the dump of what I obtain when I run this program:这是我运行此程序时获得的转储:
sh$ node --version
v14.17.1
sh$ node t.js
_write H
Trace: end
at Proxy.end (/home/sylvain/Projects/getpro/t.js:33:13)
at MyStream.onend (internal/streams/readable.js:665:10)
at Object.onceWrapper (events.js:481:28)
at MyStream.emit (events.js:375:28)
at endReadableNT (internal/streams/readable.js:1317:12)
at processTicksAndRejections (internal/process/task_queues.js:82:21)
_write e
events.js:352
throw er; // Unhandled 'error' event
^
Error [ERR_STREAM_WRITE_AFTER_END]: write after end
at writeAfterEnd (_http_outgoing.js:694:15)
at write_ (_http_outgoing.js:706:5)
at ClientRequest.write (_http_outgoing.js:687:15)
at Proxy._write (/home/sylvain/Projects/getpro/t.js:40:27)
at doWrite (internal/streams/writable.js:377:12)
at clearBuffer (internal/streams/writable.js:529:7)
at onwrite (internal/streams/writable.js:430:7)
at callback (internal/streams/writable.js:513:21)
at afterWrite (internal/streams/writable.js:466:5)
at onwrite (internal/streams/writable.js:446:7)
Emitted 'error' event on ClientRequest instance at:
at writeAfterEndNT (_http_outgoing.js:753:7)
at processTicksAndRejections (internal/process/task_queues.js:83:21) {
code: 'ERR_STREAM_WRITE_AFTER_END'
}
While investigating this issue, I noticed I can fix it for file streams by forwarding the proxy's _write
to the original object's _write
method (and not write
without underscore as showed in the code above).在调查这个问题时,我注意到我可以通过将代理的
_write
转发到原始对象的_write
方法来修复文件流(如上面的代码所示,不要在没有下划线的情况下write
)。
Unfortunately, the HTTP request object does not have a _write
method.不幸的是,HTTP 请求对象没有
_write
方法。 So I wonder if I can proxy it that way.所以我想知道我是否可以这样代理它。
MyStream
will push data as fast as possible into its internal buffer, and emit an end
event when it's done. MyStream
将尽可能快地将数据推送到其内部缓冲区,并在完成时发出end
事件。 This in turn calls Proxy.end()
, as explained here : 这又会调用
Proxy.end()
,如下所述:
By default, stream.end() is called on the destination Writable stream when the source Readable stream emits 'end', so that the destination is no longer writable.
默认情况下,当源 Readable 流发出“end”时,在目标 Writable 流上调用 stream.end(),以便目标不再可写。
However, the writable will still receive calls to _write()
because the readable's buffer may not have been drained yet.但是,可写对象仍会收到对
_write()
的调用,因为可读对象的缓冲区可能尚未耗尽。 Because you already ended the HTTP request in end()
, you get a message that you are trying to write to an ended writable.因为您已经在
end()
中结束了 HTTP 请求,所以您会收到一条消息,表明您正在尝试将其写入已结束的可写对象。
Overridingwritable._final()
is probably a better way to end the HTTP request:覆盖
writable._final()
可能是结束 HTTP 请求的更好方法:
class Proxy extends Writable {
…
_final(callback) {
this._node_req.end(callback);
}
}
The problem is that the pipe(res) is triggered on each data event.问题是每个数据事件都会触发管道(res)。 By the time your second data event occurs, the res stream is already closed.
当您的第二个数据事件发生时, res 流已经关闭。 You have to override the end method.
您必须覆盖 end 方法。 This code works:
此代码有效:
const http = require("http");
const fs = require("fs");
const { Writable, Readable } = require('stream');
class MyStream extends Readable {
constructor() {
super();
this._iterator = this.generator();
}
// __proto__;
*generator() {
for(const c of "Hello")
yield c;
}
_read(n) {
const it = this._iterator.next();
if (it.done)
this.push(null);
else
this.push(it.value);
}
}
class Proxy extends Writable {
constructor(req) {
super();
this._node_req = req;
}
// __proto__;
end(chunk, encoding, cb) {
//console.trace("end");
//this._node_req.end(chunk, encoding, cb);
return {}; // return something
}
_write(chunk, encoding, cb) {
console.log("_write", chunk.toString("utf8"));
this.prototype = new Writable();
this.prototype.constructor = Writable;
return this._node_req.write(chunk, encoding, cb);
}
_final(callback) {
this._node_req.end(callback);
}
}
const src = new MyStream()
src.pipe(new Proxy(fs.createWriteStream("out.tmp")));
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.