Using React for server-side rendering with Golang

Question

Say we want to use a Node.js process pool, to render some HTML using React. (I am not saying this is a good idea, just assume that this is the case, lulz).

Is there a way to pass a reference to the request/response streams from Golang to a Node.js process? I think the cluster module for Node.js uses this technique, by passing a file descriptor or something like that. Note that the Node.js process pool (maybe 3 processes or so), would be children of the Golang process.

Answer 1

The following is a really rough draft, that uses a channel to implement a process pool, and shows how Go's io.Reader and io.Writer interfaces can be used to plug process and HTTP streams together. The code is also on the playground , for easy copy-paste.

Note that I've written this in a hurry, just to show the general idea. Don't use this in production. I'm sure there are bugs, especially related to incomplete reads or writes. Processes exiting while idle are also not handled.

package main

import (
        "encoding/json"
        "fmt"
        "io"
        "log"
        "net/http"
        "os"
        "os/exec"
)

exec.Cmd.Stdin and exec.Cmd.Stdout are of type io.Reader and io.Writer respectively. However, it is more convenient for us to treat them the other way around. The StdinPipe and StdoutPipe methods facilitate exactly that, but they must be called only once and only before the process starts. So we store the pipes together with the command itself in a simple wrapper. This allows us to call nodeWrapper.Write([]byte) to send data to node, and nodeWrapper.Read() to read from its stdout. This is what I meant when I said in a comment you usually pass Readers and Writers around.

type nodeWrapper struct {
        *exec.Cmd
        io.Writer // stdin
        io.Reader // stdout
}

// mustSpawnNode returns a started nodejs process that executes render.js
func mustSpawnNode() nodeWrapper {
        cmd := exec.Command("node", "render.js")
        cmd.Stderr = os.Stderr

        stdin, err := cmd.StdinPipe()
        if err != nil {
                panic(err)
        }

        stdout, err := cmd.StdoutPipe()
        if err != nil {
                panic(err)
        }

        if err := cmd.Start(); err != nil {
                panic(err)
        }

        return nodeWrapper{cmd, stdin, stdout}
}

We use a simple channel-based ring buffer here to implement a process pool.

The handler parses an HTTP request and extracts information that is required to render a page. In this example we simply pass the request Path to node. We then wait for a free node process and call render. render will write directly the ResponseWriter.

func main() {
        pool := make(chan nodeWrapper, 4) // acts as a ring buffer
        for i := 0; i < cap(pool); i++ {
                pool <- mustSpawnNode()
        }

        log.Println("listening on :3000")
        log.Fatal(http.ListenAndServe(":3000", handler(pool)))
}

func handler(pool chan nodeWrapper) http.HandlerFunc {
        return func(w http.ResponseWriter, r *http.Request) {
                var renderArgs struct {
                        Path string
                }
                renderArgs.Path = r.URL.Path

                node := <-pool

                err := render(w, node, renderArgs)
                if err != nil {
                        // Assume the node process has failed and replace it
                        // with a new one.
                        node.Process.Kill()
                        pool <- mustSpawnNode()
                        http.Error(w, err.Error(), 500)
                } else {
                        pool <- node
                }
        }
}

For the rendering we a) want to pass some data to the already running node process, and b) read from node's stdout and, more importantly, have to know when to stop reading.

Usually we would set Stdout to our desired writer and simply run the process to completion. But in this case the process will not exit once it has finished rendering, so it will also not close stdout, and we need a replacement for the usual EOF signal.

This is where we have to get creative and a find a solution that works well for you. I decided on the following protocol: We write a single line of JSON encoded data to node's stdin and then decode a single JSON encoded string from node's stdout. Ideally we wouldn't buffer the whole HTML document in memory but put it directly on the wire (by writing to w in real-time). But this keeps both the Go code and render.js real simple.

func render(w io.Writer, node nodeWrapper, args interface{}) error {
        stdinErr := make(chan error, 1)
        go func() {
                stdinErr <- json.NewEncoder(node).Encode(args)
        }()

        var html string
        if err := json.NewDecoder(node).Decode(&html); err != nil {
                return err
        }
        if _, err := fmt.Fprint(w, html); err != nil {
                return err
        }

        return <-stdinErr
}

And finally, the contents of render.js:

let lineReader = require('readline').createInterface({input: process.stdin})

lineReader.on('line', (line) => {
    let data = JSON.parse(line);

    let html = "";
    html += "<h1>Path: " + data.Path + "</h1>\n";
    html += "<small>PID: " + process.pid + "</small>\n";

    process.stdout.write(JSON.stringify(html)+"\n")
})