Golang gin-gonic reverse proxying leads to panic “interface conversion: *http.timeoutWriter is not http.CloseNotifier: missing method CloseNotify”

Question

I'm using the Gin Gonic framework to create a reverse proxy endpoint, with the target endpoint being served using grpc Gateway using the code given below. This is similar to the reverse proxy methodology suggested for gin here and here

ep1 := v1.Group("/ep1")
{
    ep1.GET("/ep2", reverseProxy("http://localhost:50000"))
}

func reverseProxy(target string) gin.HandlerFunc {
    url, err := url.Parse(target)
    if err != nil {
        log.Println("Reverse Proxy target url could not be parsed:", err)
        return nil
    }
    proxy := httputil.NewSingleHostReverseProxy(url)
    return func(c *gin.Context) {
        proxy.ServeHTTP(c.Writer, c.Request)
    }
}

However, when on actually sending a request to this gin endpoint (/ep1/ep2) a go panic is seen:

interface conversion: *http.timeoutWriter is not http.CloseNotifier: missing method CloseNotify
/usr/local/Cellar/go/1.8/libexec/src/runtime/panic.go:489 (0x10288df)
    gopanic: reflectcall(nil, unsafe.Pointer(d.fn), deferArgs(d), uint32(d.siz), uint32(d.siz))
/usr/local/Cellar/go/1.8/libexec/src/runtime/iface.go:131 (0x100c3af)
    additab: panic(&TypeAssertionError{"", typ.string(), inter.typ.string(), iname})
/usr/local/Cellar/go/1.8/libexec/src/runtime/iface.go:79 (0x100bc34)
    getitab: additab(m, true, canfail)
/usr/local/Cellar/go/1.8/libexec/src/runtime/iface.go:256 (0x100cbb8)
    assertI2I: r.tab = getitab(inter, tab._type, false)
/path/to/vendor/github.com/gin-gonic/gin/response_writer.go:110 (0x14de6f3)
    (*responseWriter).CloseNotify: return w.ResponseWriter.(http.CloseNotifier).CloseNotify()
/usr/local/Cellar/go/1.8/libexec/src/net/http/httputil/reverseproxy.go:142 (0x14d4d12)
    (*ReverseProxy).ServeHTTP: notifyChan := cn.CloseNotify()
/path/to/main.go:379 (0x16d2ead)
    reverseProxy.func1: proxy.ServeHTTP(c.Writer, c.Request)
/path/to/vendor/github.com/gin-gonic/gin/context.go:97 (0x14d657a)
    (*Context).Next: c.handlers[c.index](c)
/path/to/middlewares/locale.go:12 (0x15737d9)
    getLocaleMiddleware.func1: c.Next()
/path/to/vendor/github.com/gin-gonic/gin/context.go:97 (0x14d657a)
    (*Context).Next: c.handlers[c.index](c)
/path/to/middlewares/session_cookie.go:27 (0x1574e7c)
    getSessionCookieMiddleware.func1: c.Next()
/path/to/vendor/github.com/gin-gonic/gin/context.go:97 (0x14d657a)
    (*Context).Next: c.handlers[c.index](c)
/path/to/middlewares/affiliate_api.go:27 (0x15729a1)
    getAffiliateAPIMiddleware.func1: c.Next()
/path/to/vendor/github.com/gin-gonic/gin/context.go:97 (0x14d657a)
    (*Context).Next: c.handlers[c.index](c)
/path/to/middlewares/metrics.go:17 (0x157465b)
    getMetricsMiddleware.func1: c.Next()
/path/to/vendor/github.com/gin-gonic/gin/context.go:97 (0x14d657a)
    (*Context).Next: c.handlers[c.index](c)
/path/to/middlewares/input_validations.go:75 (0x1572dcb)
    getInputValidationMiddleware.func1: c.Next()
/path/to/vendor/github.com/gin-gonic/gin/context.go:97 (0x14d657a)
    (*Context).Next: c.handlers[c.index](c)
/path/to/middlewares/logger.go:68 (0x1573aea)
    LoggerWithWriter.func1: c.Next()
/path/to/vendor/github.com/gin-gonic/gin/context.go:97 (0x14d657a)
    (*Context).Next: c.handlers[c.index](c)
/path/to/middlewares/request_tracer.go:13 (0x1574d6c)
    getTracerContext.func1: c.Next()
/path/to/vendor/github.com/gin-gonic/gin/context.go:97 (0x14d657a)
    (*Context).Next: c.handlers[c.index](c)
/path/to/vendor/github.com/gin-gonic/gin/recovery.go:45 (0x14e4b6a)
    RecoveryWithWriter.func1: c.Next()
/path/to/vendor/github.com/gin-gonic/gin/context.go:97 (0x14d657a)
    (*Context).Next: c.handlers[c.index](c)
/path/to/vendor/github.com/gin-gonic/gin/gin.go:284 (0x14dc710)
    (*Engine).handleHTTPRequest: context.Next()
/path/to/vendor/github.com/gin-gonic/gin/gin.go:265 (0x14dc02b)
    (*Engine).ServeHTTP: engine.handleHTTPRequest(c)
/usr/local/Cellar/go/1.8/libexec/src/net/http/server.go:2967 (0x140fa53)
    (*timeoutHandler).ServeHTTP.func1: h.handler.ServeHTTP(tw, r)
/usr/local/Cellar/go/1.8/libexec/src/runtime/asm_amd64.s:2197 (0x1054851)

Any ideas on why this might be happening or what's wrong in the code?

Answer 1

It was found that the issue was seen because the codebase wasn't using the Run() method from gin-gonic directly. Instead, it was using a timeout when starting an http server as follows (using partial, relevant code here):

type H struct {
    sync.Mutex
    Engine   *gin.Engine
    listener net.Listener
    running  bool
}
.
.
.
var h H
s := &http.Server{
    Addr:         address,
    Handler:      http.TimeoutHandler(h.Engine, time.Duration(100000)*time.Millisecond, ""),
    ReadTimeout:  time.Duration(100000) * time.Millisecond,
    WriteTimeout: time.Duration(100000) * time.Millisecond,
}


h.listener, err := net.Listen("tcp", s.Addr)
if err != nil {
    return err
}

h.running = true
s.Serve(h.listener)

However, http.TimeoutHandler doesn't implement the http.CloseNotifer interface as mentioned at http://grokbase.com/t/gg/golang-dev/13796p5h1n/net-http-timeouthandler-vs-closenotify This resulted in a panic with the error message interface conversion: *http.timeoutWriter is not http.CloseNotifier: missing method CloseNotify

Therefore, as a workaround for this issue, the Server Handler was modified to be the gin engine directly while using the ReadTimeout and WriteTimeout values of http.Server for timeout purposes.

Modified code which no longer panics, and results in successful reverse-proxying:

type H struct {
    sync.Mutex
    Engine   *gin.Engine
    listener net.Listener
    running  bool
}
.
.
.
var h H
s := &http.Server{
    Addr:         address,
    Handler:      h.Engine,
    ReadTimeout:  time.Duration(100000) * time.Millisecond,
    WriteTimeout: time.Duration(100000) * time.Millisecond,
}

h.listener, err := net.Listen("tcp", s.Addr)
if err != nil {
    return err
}

h.running = true
s.Serve(h.listener)

Note that only the Handler for &http.Server needed to be modified here. Also, no modification to the original reverse proxy code from the question was needed.