Golang type conversion/assertion issue with unmarshalling json

Question

package main

import (
    "fmt"
    "encoding/json"
    "reflect"
)

type GeneralConfig map[string]interface{}

var data string = `
{
    "key":"value",
    "important_key":
        {"foo":"bar"}
}`

func main() {
    jsonData := &GeneralConfig{}
    json.Unmarshal([]byte(data), jsonData)

    fmt.Println(reflect.TypeOf(jsonData)) //main.GeneralConfig

    jsonTemp := (*jsonData)["important_key"]
    fmt.Println(reflect.TypeOf(jsonTemp)) //map[string]interface {}

    //newGeneralConfig := GeneralConfig(jsonTemp)
    //cannot convert jsonTemp (type interface {}) to type GeneralConfig:
    //need type assertion

    newGeneralConfig := jsonTemp.(GeneralConfig)
    //fmt.Println(reflect.TypeOf(newGeneralConfig))
    //panic: interface conversion: interface {} is map[string]interface {},
    //not main.GeneralConfig

}

Available at the playground

I understand that I can use a nested struct in lieu of GeneralConfig , but that would require me knowing the exact structure of the payload, ie it wouldn't work for different keys (I would be locked into "important_key").

Is there a golang workaround for when I don't know what the value of "important_key" is? I say golang, because if possible, one could require all "important_keys" to have a constant parent key, which could resolve this issue.

To summarize, given an arbitrary json object, there must be a way that I can traverse its keys, and if a value is a custom type, convert the value to that type. Right now it seems that if I use type conversion, it tells me that the type is interface{} and I need to use type assertion; however, if I use type assertion, it tells me that interface{} is map[string]interface{} not main.GeneralConfig .

Answer 1

I agree the comments about trying to utilise the expected structure of the incoming JSON in order to write well-defined Structs, but I'll attempt to answer the question anyway.

The thing to take away from what you're seeing printed versus the error messages that you're seeing is that the compiler knows less about the type than the runtime because the runtime can look at the actual value. To bring the compiler up-to-speed we must (i) assert (*jsonData)["important_key"] is a map[string]interface{} -- the compiler only knows it to be an interface{} -- and then (ii) type-cast that to a GeneralConfig type. See:

package main

import (
    "fmt"
    "encoding/json"
)

type GeneralConfig map[string]interface{}

func main() {
    jsonStruct := new(GeneralConfig)
    json.Unmarshal([]byte(`{"parent_key": {"foo": "bar"}}`), jsonStruct)
    fmt.Printf("%#v\n", jsonStruct)
    // => &main.GeneralConfig{"parent_key":map[string]interface {}{"foo":"bar"}}

    nestedStruct := (*jsonStruct)["parent_key"]
    fmt.Printf("%#v\n", nestedStruct)
    // => map[string]interface {}{"foo":"bar"}
    // Whilst this shows the runtime knows its actual type is
    // map[string]interface, the compiler only knows it to be an interface{}.

    // First we assert for the compiler that it is indeed a
    // map[string]interface{} we are working with. You can imagine the issues
    // that might arrise if we has passed in `{"parent_key": 123}`.
    mapConfig, ok := nestedStruct.(map[string]interface{})
    if !ok {
        // TODO: Error-handling.
    }

    // Now that the compiler can be sure mapConfig is a map[string]interface{}
    // we can type-cast it to GeneralConfig:
    config := GeneralConfig(mapConfig)
    fmt.Printf("%#v\n", config)
    // => main.GeneralConfig{"foo":"bar"}
}

Answer 2

You are looking for json.RawMessage.
You can delay unmarshalling based upon some other value and then force it to unmarshal to a specific type.

This is not a good idea, but might be closer to what you are looking for.

http://play.golang.org/p/PWwAUDySE0

Answer 3

This is a standard "workaround" if get what you're after. When handling unknown data you can implement this pattern (modified from your example) of switching on the type recursively to get to the concrete values in an unknown body of json data.

package main

import (
    "encoding/json"
    "fmt"
    "reflect"
)

var data = `
{
    "key":"value",
    "important_key":
        {"foo":"bar"}
}`

func main() {
    var jsonData interface{}
    json.Unmarshal([]byte(data), &jsonData)

    fmt.Println(reflect.TypeOf(jsonData))

    parseArbitraryJSON(jsonData.(map[string]interface{}))
}

func parseArbitraryJSON(data map[string]interface{}) {
    for k, v := range data {
        switch a := v.(type) {
        case string:
            fmt.Printf("%v:%v\n", k, a)
        case map[string]interface{}:
            fmt.Printf("%v:%v\n", k, a)
            parseArbitraryJSON(a)
        }
    }
}

The resulting output is:

map[string]interface {}
key:value
important_key:map[foo:bar]
foo:bar

This example only accounts for the base data being a string type but you can switch on any type that you expect to receive, and like any switch you can group your cases, so you can treat all numbers similarly for example.