How to avoid last blank line in Deno TextLineStream

Question

I run the following code,

import { TextLineStream } from "https://deno.land/std@0.136.0/streams/mod.ts";
const cmd1 = new Deno.Command("echo", {args:["foo\n\nbar"], stdout: "piped"}).spawn();
for await (const chunk of cmd1.stdout.pipeThrough(new TextDecoderStream).pipeThrough(new TextLineStream)) {
    console.log("result: ", chunk);
}

got

result:  foo
result:
result:  bar
result:

want

result:  foo
result:
result:  bar

How can I detect the end of a stream and avoid a blank line at the end?
Thank you for your help.

Answer 1

The type of cmd1.stdout in your question is ReadableStream<Uint8Array> . After piping this through the other transform stream classes, the resulting value is ReadableStream<string> . When using a for await...of loop, each yielded value is simply a line ( string ), so you don't have an intrinsic reference for determining the end of stream within the loop in order to make decisions. (This would still be true even if you were to manually iterate the stream's AsyncIterableIterator<string> .) Once the stream ends and the loop is done, it's already too late to go back and un-handle the final (empty) line.

However, you can deal with this by storing each previous line in a variable outside the loop, and using that previous line inside the loop (instead of the current line). Then, after the loop ends, check to see whether or not the final previous line is empty — if it isn't, just use it like you did the other lines — if it is, then you can just ignore it.

Below I've included a self-contained example that you can run on any platform without permissions and see the same results — it's based on the data in your question and compares the exact input you showed with a couple of similar inputs with different kinds of endings for juxtaposition. It also includes line numbers as a bonus — they are helpful as a reference mechanism when comparing against the default behavior of the TextLineStream . The interesting code is in the RefinedTextLineStream class, and you can simply use it in your own code like this to see the desired output that you described:

 const stream = cmd1.stdout.pipeThrough(new TextDecoderStream()).pipeThrough(new TextLineStream()).pipeThrough(new RefinedTextLineStream()); for await (const [line] of stream) { console.log("result: ", line); }

---

Here's the reproducible example:

so-74905946.ts :

import { TextLineStream } from "https://deno.land/std@0.170.0/streams/text_line_stream.ts";

type LineWithNumber = [line: string, lineNumber: number];

/** For use with Deno's std library TextLineStream */
class RefinedTextLineStream extends TransformStream<string, LineWithNumber> {
  #lineNumber = 0;
  #previous = "";

  constructor() {
    super({
      transform: (textLine, controller) => {
        if (this.#lineNumber > 0) {
          controller.enqueue([this.#previous, this.#lineNumber]);
        }
        this.#lineNumber += 1;
        this.#previous = textLine;
      },
      flush: (controller) => {
        if (this.#previous.length > 0) {
          controller.enqueue([this.#previous, this.#lineNumber]);
        }
      },
    });
  }
}

const exampleInputs: [name: string, input: string][] = [
  ["input from question", "foo\n\nbar\n"],
  ["input with extra final line feed", "foo\n\nbar\n\n"],
  ["input without final line feed", "foo\n\nbar"],
];

for (const [name, input] of exampleInputs) {
  console.log(`\n${name}: ${JSON.stringify(input)}`);

  const [textLineStream1, textLineStream2] = new File([input], "untitled")
    .stream()
    .pipeThrough(new TextDecoderStream())
    .pipeThrough(new TextLineStream())
    .tee();

  console.log("\ndefault:");
  let lineNumber = 0;
  for await (const line of textLineStream1) {
    lineNumber += 1;
    console.log(lineNumber, line);
  }

  console.log("\nskipping empty final line:");
  const stream = textLineStream2.pipeThrough(new RefinedTextLineStream());
  for await (const [line, lineNumber] of stream) console.log(lineNumber, line);
}

Running in the terminal:

% deno --version
deno 1.29.1 (release, x86_64-apple-darwin)
v8 10.9.194.5
typescript 4.9.4

% deno run so-74905946.ts  

input from question: "foo\n\nbar\n"

default:
1 foo
2 
3 bar
4 

skipping empty final line:
1 foo
2 
3 bar

input with extra final line feed: "foo\n\nbar\n\n"

default:
1 foo
2 
3 bar
4 
5 

skipping empty final line:
1 foo
2 
3 bar
4 

input without final line feed: "foo\n\nbar"

default:
1 foo
2 
3 bar

skipping empty final line:
1 foo
2 
3 bar