如何在Go中通过HTTP解组JSON

Question

I am accessing an API that returns JSON in the form:

[{"UniqueID":1234, "DocID":5678}, {"UniqueID":5678, "DocID":9101112}]

this API was written in Go and that sample output is how the return is displayed in a browser. the content-type header is application/json

I have the following code to retrieve and unmarshal this:

    type UniqueIDDocID struct{
        UniqueID int64 `json: "UniqueID"`
        DocID int64 `json: "DocID"`
    }
    type UniqueIDDocIDCollection struct{
        FullList []UniqueIDDocID
    }

    func retrieveUniqueIDByPublication(nodeid int64, publication string){
        // call the API
        //hgr := new(retrieval.HttpGetRetrieval)
// endpoint is defined here - I have removed for privacy reasons
        fmt.Println(endpoint) // the url which I know works
        /*response, err := hgr.RequestResponse(endpoint)
            if err != nil {
                l4g.Warn("Could not retrieve the endpoint %s. Error: ", endpoint, err)
                return
            }
            defer hgr.CloseResponse(response)
            queryResult := hgr.ReadResponse(response)*/
            client := new(http.Client)
            request, err := http.NewRequest("GET", endpoint, nil)
            if err != nil {
                l4g.Warn("Could not retrieve the endpoint %s. Error: %v", endpoint, err)
                return  
            }
            request.Header.Set("Content-Type", "applicaiton/json")
            response, err := client.Do(request)
            if err != nil {
                l4g.Warn("Could not retrieve the endpoint %s. Error: %v", endpoint, err)
                return  
            }
            defer response.Body.Close()
            var reader io.ReadCloser
            reader = response.Body
            defer reader.Close()
            queryResult, readErr := ioutil.ReadAll(reader)
            if err != nil {
                l4g.Warn("Could not retrieve the endpoint %s. Error: %v", endpoint, readErr)
                return  
            }
            if queryResult == nil {
                l4g.Warn("Nothing returned %s.", endpoint)
                return
            }


            var details UniqueIDDocIDCollection
        // process return
            if err:=json.Unmarshal(queryResult, &details); err != nil{
                l4g.Warn("Failed to unmarshall return %v from %s", queryResult, endpoint)
                return
            }
            writeUniqueIDFile(details)
    }

I am getting the "Failed to Unmarshall" message and the details in the log shows things like:

[91 123 34 85 110 105 113 117 101 73 68 34 58 34 56 51     57 51 50 53 56 54 34 44 34 68 111 99 73 68 34 58 52 49 50 49 54 57 49 57 125 44 123 34 85 110 105 113 117 101]

As a representative sample.

What am I doing wrong in this unmarshall step?

My desired output is that the UniqueIDDocIDCollection structu has a slice on it that contains items of type UniqueIDDocID that I can then take the UniqueID from and write it to a line delimited file.

I have been Googling around and have tried a number of things but each time this is what I get.

If you have any suggestions about the source JSON too then share those as I may be able to make the changes in the API.

Thanks for the help in advance Nathan

Answer 1

The error data is a []byte slice; you will likely get something more useful if you Printf it to a %s ( fmt.Printf("foo: %s\\n", foo) ) or wherever you have it now wrap it in string(foo) .

I made a simpler example of your Unmarshall, and it seems to work fine. I suspect the problem is with the input data after all?

package main

import (
    "encoding/json"
    "log"
)

type UniqueIDDocID struct {
    UniqueID int64 `json: "UniqueID"`
    DocID    int64 `json: "DocID"`
}
type UniqueIDDocIDCollection struct {
    FullList []UniqueIDDocID
}

const INPUT = `[{"UniqueID":1234, "DocID":5678}, {"UniqueID":5678, "DocID":9101112}]`

func main() {

    coll := new(UniqueIDDocIDCollection)

    err := json.Unmarshal([]byte(INPUT), &coll.FullList)

    if err != nil {
        log.Printf("Could not unmarshall %s: %s", INPUT, err)
    }

    log.Printf("Now have data: %#v\n", coll)

}

Outputs ( try it on play.golang )

Now have data: &main.UniqueIDDocIDCollection{FullList:[]main.UniqueIDDocID{main.UniqueIDDocID{UniqueID:1234, DocID:5678}, main.UniqueIDDocID{UniqueID:5678, DocID:9101112}}}

Start by changing %v to %s in the "failed to unmarshall" error message; maybe you don't have the data you are expecting. You should also include the error returned from json.Unmarshall, it might tell you what's going wrong. :-)