How to mock http.NewRequest to return an error

Question

I have a Go method that makes an http request to an external web page, that I am screen scraping. I have unit tests for when everything works as planned. I can mock the httpclient and set the response to a predefined value. However, I am having trouble creating tests for when things don't work. I want to mock http.NewRequest to return an error.

I am using httptest and go tool cover.

Method to test:

func (cli *Client) GetSomething(sometext string) (string, error) {

    searchURL := strings.Replace(someUrl, "{PERSON_NAME}", url.QueryEscape(sometext), 1)
    req, err := http.NewRequest("GET", someUrl, nil)
    if err != nil {
    // I want to test this scenario
        return "", errors.Wrap(err, "failed to build request")
    }

    resp, err := cli.HTTPClient.Do(req)

    if err != nil {
        return "", errors.Wrap(err, "request failed")
    }

    defer resp.Body.Close()

    document, err := goquery.NewDocumentFromReader(resp.Body)

    if err != nil {
        return "", err
    }

    value:= document.Find(".someclass").Text()

    return value, nil
}

Test Method works for the httpclient. Not sure how to mock http.NewRequest to return an error. This would probably never happen, but just to be complete and as an exercise.

func TestGetSomethingFailedReqest(t *testing.T) {

    h := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        panic("Create error")
    })

//TestingHTTPClient is a helper function to create an HTTPClient.  Not needed for this test.

    httpClient, teardown := test.TestingHTTPClient(h)  
    defer teardown()
    scraperClient := NewClient()
    scraperClient.HTTPClient = httpClient

    var person= "John Doe"

    _, err := scraperClient.GetSomething(person)

    assert.Regexp(t, ".*failed to build request", err.Error())
}

Answer 1

If you want your tests to be reliable you should try to avoid mocking as much as you can.

So if you are wondering how you can test a library that is creating http request, the best way is to create a local server in your unit test, this way you can test every aspects of the request.

The best example I can provide you is this: https://github.com/gojek/heimdall/blob/master/httpclient/client_test.go In Heimdall an open source http client, you can see how they implemented the tests without mocks.

And for your specific question, BentCoder replied to you: "just use "*?" as request method. This should return net/http: invalid method "*?" error. "

Answer 2

Jeremie's answer regarding the specific question is correct, but the request method could be hardcoded.

The way http code is usually tested is by parameterising the url sent, so that it can be replaced in tests with a fake server.

Here's a trick for that scenario: require the url parameter to contain a trailing slash, and omit it in the test.

func main() {
    ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        fmt.Fprintln(w, "hello")
    }))
    
    err := codeToTest(ts.URL + "/")
    fmt.Println(err) // Works!
    
    err = codeToTest(ts.URL) // <- no "/" at end of URL
    fmt.Println(err) // Fails!
}

func codeToTest(url string) error {
    _, err := http.NewRequest("GET", fmt.Sprintf("%vendpoint", url), nil)
    if err != nil {
        return err
    }
    return nil
}

Go playground

https://play.golang.org/p/JQBudQPthdE