[英]Node.js Writable Stream Finish Not Firing In Class
I am trying to create a CSV class that can be used inside other scripts in my application. 我正在尝试创建一个可以在我的应用程序的其他脚本中使用的CSV类。 When the CSV class is instantiated, it creates a writable stream to a file specified by the user.
实例化CSV类后,它将创建可写流到用户指定的文件。 The write and destroy methods seem to work, but I can't figure out how to get the 'writeEnd' member on the class to execute once the data has finished writing to the file.
写入和销毁方法似乎可行,但是我无法弄清楚一旦数据完成写入文件后如何让类上的'writeEnd'成员执行。
The 'writeEnd' member variable is a function that should be overridden by the user. “ writeEnd”成员变量是一个应由用户覆盖的函数。 For example, here is a file where I am testing out the classes functionality, and overriding 'writeEnd' to be a function of my own choosing.
例如,这是一个文件,我在其中测试类的功能,并将“ writeEnd”重写为我自己选择的功能。
PS Please see the question in bold at the bottom! PS:请在底部以粗体显示该问题!
const CSV = require('./shared/classes/csv');
const csv = new CSV(__dirname);
csv.writeEnd = () => {
console.log('Finished!');
};
for (let i = 0; i < 1000000; i++) {
csv.write('Hello World.');
}
I was hoping for 'Finished!' 我希望“完成!” to be logged to the console, but the function does not fire at all.
记录到控制台,但该功能根本不会触发。 I hope I am doing something wrong that someone can catch pretty easily.
我希望我做错了某人可以很容易地抓住的东西。
For your reference, here is the class file untouched: 供您参考,这是未修改的类文件:
const { createWriteStream } = require('fs');
const { Readable } = require('stream');
/**
* @class CSV
*/
module.exports = class CSV {
constructor(path) {
this.readStream = new Readable({ read() {} });
this.writeStream = createWriteStream(`${path}/csv/data.csv`);
this.readStream.pipe(this.writeStream);
this.writeEnd = () => {};
}
/**
* @method write
* @param {any} data
*/
write(data) {
this.readStream.push(`${data}\n`);
}
/**
* @method destroy
*/
destroy() {
this.readStream.destroy();
this.writeStream.destroy();
}
};
Below, is one of my failed attempts: 以下是我失败的尝试之一:
/**
* @class CSV
*/
module.exports = class CSV {
constructor(path) {
this.readStream = new Readable({ read() {} });
this.writeStream = createWriteStream(`${path}/csv/data.csv`);
this.readStream.pipe(this.writeStream);
// I'm wondering if this executes immediately because no writing is taking place
// during instantiation
this.writeStream.on('finish', this.writeEnd);
this.writeEnd = () => {};
}
/**
* @method write
* @param {any} data
*/
write(data) {
this.readStream.push(`${data}\n`);
}
/**
* @method destroy
*/
destroy() {
this.readStream.destroy();
this.writeStream.destroy();
}
};
I am wondering if I need to actually listen for the very first time the readStream gets data pushed to it, then set the 'finish' callback? 我想知道我是否真的需要在readStream第一次将数据推送到它时才真正侦听,然后设置“完成”回调?
The problem is that the custom/overriden writeEnd-method is never called, since the event-emitter keeps a reference to the original handler, ie the function you set in your constructor: this.writeEnd = () => {};
问题在于,永远不会调用自定义/重写的writeEnd方法,因为事件发射器会保留对原始处理程序(即您在构造函数中设置的函数)的引用:
this.writeEnd = () => {};
The easiest way is to allow to pass a callback function to the constructor of the CSV
-class and use this as a finish
-handler. 最简单的方法是允许将回调函数传递给
CSV
-class的构造函数,并将其用作finish
-handler。 Consider this simple example: 考虑以下简单示例:
const EventEmitter = require("events").EventEmitter;
class CSV {
constructor(customWriteEndCb) {
this.writeEnd = () => {
console.log("in original writeEnd");
};
this.writeEnd = customWriteEndCb || this.writeEnd;
this.writeStream = new EventEmitter();
this.writeStream.on('finished', this.writeEnd);
}
testMe() {
this.writeStream.emit('finished');
}
}
const customWriteEnd = () => {
console.log("in custom writeEnd")
}
const csv = new CSV(customWriteEnd);
csv.testMe(); // will print "in custom writeEnd"
Correct me if I'm wrong but here is a minimal working example: 如果我错了,请纠正我,但这是一个最小的工作示例:
const { createWriteStream } = require('fs');
class CSV {
constructor(path) {
this.writeStream = createWriteStream(`${path}/csv/data.csv`);
this.writeEnd = () => {};
}
write(data) {
this.writeStream.write(`${data}\n`)
}
end() {
this.writeStream.end()
this.writeStream.on('finish', this.writeEnd)
}
};
const csv = new CSV(__dirname);
csv.writeEnd = () => console.log('Finished')
for (let i = 0; i < 1000000; i++) {
csv.write('Hello World.');
}
csv.end()
I removed the readable stream that I find unnecessary, and the destroy shouldn't be called. 我删除了我认为不必要的可读流,并且不应该调用destroy。
Use end() instead of destroy if data should flush before close
如果数据应在关闭之前刷新,请使用end()而不是destroy
https://nodejs.org/api/stream.html#stream_writable_destroy_error https://nodejs.org/api/stream.html#stream_writable_destroy_error
You can see a Finished at the end. 您可以在最后看到“完成”。
I'm not sure why are you using classes here, anyway: 无论如何,我不确定为什么要在这里使用类:
const fs = require('fs');
class CSV
{
constructor(path) {
this._ws = fs.createWriteStream(`${path}/csv/data.csv`);
}
write(data) {
this._ws.write(`${data}\n`);
}
close() {
const _this = this;
return new Promise(function (resolve, reject) {
_this._ws.once('finish', resolve);
_this._ws.once('error', reject);
_this._ws.end();
});
}
}
async function main()
{
const csv = new CSV('path1');
for (let i = 0; i < 1000000; ++i) {
csv.write(`chunk ${i}`);
}
await csv.close();
console.log('end');
}
function panic(error)
{
console.error(error);
process.exit(1);
}
// https://stackoverflow.com/a/46916601/1478566
main().catch(panic).finally(clearInterval.bind(null, setInterval(a=>a, 1E9)));
And the following is approach without classes: 以下是没有类的方法:
const fs = require('fs');
async function main()
{
const ws = fs.createWriteStream('a.txt');
for (let i = 0; i < 1000000; ++i) {
ws.write(`chunk ${i}\n`);
}
ws.end();
await promise_from_stream(ws);
console.log('end');
}
function panic(error)
{
console.error(error);
process.exit(1);
}
function promise_from_stream(stream)
{
/**
* https://stackoverflow.com/a/34310963/1478566
* > end and finish are the same event BUT on different types of
* > Streams.
* > * stream.Readable fires ONLY end and NEVER finish
* > * stream.Writable fires ONLY finish and NEVER end
*/
return new Promise(function (resolve, reject) {
stream.once('end', resolve);
stream.once('finish', resolve);
stream.once('error', reject);
});
}
// https://stackoverflow.com/a/46916601/1478566
main().catch(panic).finally(clearInterval.bind(null, setInterval(a=>a, 1E9)));
So it was through a group effort of different answers and comments that landed me on a simple solution! 因此,正是通过不同答案和评论的共同努力,我才找到了一个简单的解决方案! Thank you very much to everyone who took the time to share their advice.
非常感谢所有抽出宝贵时间分享建议的人。
I removed the readable stream, as that was completely unnecesssary, and simply used the write method on the writable stream. 我删除了可读流,因为它完全没有必要,只对可写流使用了write方法。 I also pass a callback function to the constructor upon instantiation.
实例化后,我还将回调函数传递给构造函数。
Here is my final code: 这是我的最终代码:
const { createWriteStream } = require('fs');
/**
* @class CSV
*/
module.exports = class CSV {
constructor(path, cb) {
this.writeStream = createWriteStream(`${path}/csv/data.csv`);
this.writeStream.on('finish', cb);
}
/**
* @method write
* @param {any} data
*/
write(data) {
this.writeStream.write(`${data}\n`);
}
/**
* @method end
*/
end() {
this.writeStream.end();
}
};
And the test file: 和测试文件:
const CSV = require('./shared/classes/csv');
const csv = new CSV(__dirname, cb);
function cb() {
console.log('You win sir.');
}
for (let i = 0; i < 1000000; i++) {
csv.write('Hello World.');
}
csv.end();
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.