管道传输到可写流时暂停可读流

Question

I have a readable stream that I would like to pause. It is piped to a writable stream. My code looks like this

const { get } = require('https');
const { createWriteStream };

const writableStream = createWriteStream(SOME_PATH);
get(SOME_URL, (res) => {
  res.pipe(writableStream);
  setTimeout(() => {
    res.pause();
  }, 2000);

  setTimeout(() => {
    res.resume();
  }, 4000);
});

This works well on Mac. But for some reason, on Windows while downloading from an https URL, this doesn't pause.

I think this is because my readable stream is piped to a writable stream and the writable stream is asking for more data, which resumes the stream. If I unpipe, this will solve the issue. Here is my code when I unpipe

const writableStream = createWriteStream(SOME_PATH);
get(SOME_URL, (res) => {
  res.pipe(writableStream);
  setTimeout(() => {
    res.unpipe(writableStream);
    res.pause();
  }, 2000);

  setTimeout(() => {
    res.pipe(writableStream);
    res.resume();
  }, 4000);
});

This actually causes my download to pause. But this creates a new issue. After calling res.unpipe() , I still get data events. This means that the few milliseconds in between calling res.unpipe() and res.pause() , some of my data is sent to the res pipe, but not written to writableStream pipe. This ends with my downloaded file getting corrupted.

Is there any way to fix this problem? I'm not married to the idea of unpiping, it's just the only solution I could come up with.

I was thinking about storing the data the res gets when not piped to writableStream and manually passing it to writableStream when they pipe again. Is that even possible? If not, are there other ways I can go about pausing a stream when piped to a readable stream?

Answer 1

I figured it out. I'm not sure why this solution works, but it works for me. Instead of unpiping before I paused, I created a listener for pause and unpipe there. In addition, I also set res.readableFlowing to false manually. With these two additions, I was able to pause and resume without the download file breaking. Here's the implementation

let isPaused = false;
const writableStream = createWriteStream(SOME_PATH);
get(SOME_URL, (res) => {
  res.pipe(writableStream);
  setTimeout(() => {
    isPaused = true;
    res.pause();
  }, 2000);

  setTimeout(() => {
    isPaused = false;
    res.pipe(writableStream);
    res.resume();
  }, 4000);

  res.on('pause', () => {
    // Add this flag because pause event gets called many times, and we only
    // want this code to happen when we call it manually
    if (isPaused) {
      res.unPipe(writableStream);
      res.readableFlowing = false;
    }
  });
});