如何克隆可讀流

Question

我有一個流，我試圖將同一個流提交到兩個不同的目的地。 第一個目的地是 AWS S3，第二個目的地是通過 http 請求到達某個其他后端。

const document = fs.createReadStream(process.cwd() + "/test/resources/" + "id/document.jpg");

const s3Response = await submitToS3(document);

const backendResponse = await submitToBackend(document);

據我所知，一個流只能被讀取一次。 如何將相同的流發送到兩個不同的目的地。

我想克隆流，但只是創建一個新變量並將流分配給該變量是行不通的。

Answer 1

你可以檢查這個 npm 模塊： https ://www.npmjs.com/package/readable-stream-clone

npm install readable-stream-clone

const fs = require("fs");
const ReadableStreamClone = require("readable-stream-clone");
 
const readStream = fs.createReadStream('text.txt');
 
const readStream1 = new ReadableStreamClone(readStream);
const readStream2 = new ReadableStreamClone(readStream);
 
const writeStream1 = fs.createWriteStream('sample1.txt');
const writeStream2 = fs.createWriteStream('sample2.txt');
 
readStream1.pipe(writeStream1)
readStream2.pipe(writeStream2)

Answer 2

Raghavendra 的回答暗示了一個很好的潛在方向。 您可以將多個管這個答案與S3管道實現這個答案。

對於submitToBackend部分，不確定您的實現究竟是什么樣子，但假設您可以通過管道傳輸某種 HTTP 請求......

例子：

var fs = require("fs");
const request = require("request");
const AWS = require("aws-sdk");
const s3 = new AWS.S3();

const rs = fs.createReadStream(process.cwd() + "/test/resources/" + "id/document.jpg");

function uploadFromStream(s3) {
  var pass = new stream.PassThrough();

  var params = {Bucket: BUCKET, Key: KEY, Body: pass};
  s3.upload(params, function(err, data) {
    console.log(err, data);
  });

  return pass;
}

rs.pipe(uploadFromStream(s3));

// Just guessing at your submitToBackend implementation:
const backendWs = request.post("http://example.com/docs");

// However it works, if you can get to a stream.Writable, you can now pipe the same stream.Readable:
rs.pipe(backendWs);

Answer 3

var fs = require("fs");
var ReadableStreamClone = require("readable_stream");

var readStream = fs.createReadStream('text1.txt');

var readStream1 = new ReadableStreamClone(readStream);
var readStream2 = new ReadableStreamClone(readStream);

var writeStream1 = fs.createWriteStream('testsample1.txt');
var writeStream2 = fs.createWriteStream('testsample2.txt');

readStream1.pipe(writeStream1)
readStream2.pipe(writeStream2)