Idiomatic way to convert from a scanner to a io.Reader

Question

I recently ran into this issue of how to read from a CSV file, apply some transformation to every line and write to a HTTP request.

The problem I was faced with was how to convert from a line-by-line reader which returns an arbitrary number of bytes (like a Scanner) to a byte reader, which returns a fixed amount of bytes at every call to Read().

The best solution I came up with is to build a custom io.Reader that would read from the Scanner and buffer bytes locally if they wouldn't fit. Then the buffered bytes would be returned on the next call to Read().

This is what I came up with: https://gist.github.com/paulsc/6a0bf30a2a8d898f7a8086aedf6af1e1

Intuitively, this feels like the wrong solution, because the code seems like a fairly low-level solution that might already be in the standard library.

Is there a better way, more idiomatic to do this with standard go components ?

Answer 1

A simple method is using io.Pipe .

func ScannerToReader(scanner *bufio.Scanner) io.Reader {
    reader, writer := io.Pipe()


    go func() {
        defer writer.Close()
        for scanner.Scan() {
            writer.Write(scanner.Bytes())
        }
    }()

    return reader
}