Reading and streaming responses from http servers that are using chunk transport in go

Question

I have an http server that responds to requests via the chunk protocol. In this simple example, it responds with the time every second. I have another server that forwards requests to this time server and respond with the same chunk protocol.

curl -i localhost:8000 returns the right thing

however curl -i localhost:9000 returns a chunked response but only after three seconds (not every second). This means that somehow io.copy does not really flush the output as soon as it comes

Answer 1

io.Copy has to copy the entire contents before you call Flush . If you want the contents sent before the buffer has filled, you need to call flush after each write yourself.

A complete version of io.Copy that flushes an http.ResponseWriter after each write would look like:

func flushCopy(dst io.Writer, src io.Reader) (written int64, err error) {
    buf := make([]byte, 1024 * 8)
    flusher, canFlush := dst.(http.Flusher)
    for {
        nr, er := src.Read(buf)
        if nr > 0 {
            nw, ew := dst.Write(buf[0:nr])
            if nw > 0 {
                if canFlush {
                    flusher.Flush()
                }
                written += int64(nw)
            }
            if ew != nil {
                err = ew
                break
            }
            if nr != nw {
                err = io.ErrShortWrite
                break
            }
        }
        if er == io.EOF {
            break
        }
        if er != nil {
            err = er
            break
        }
    }
    return written, err
}