Working with JSON Nested in JSON Using Python/Pandas

Question

I am trying to load JSON data using python, however, it looks like this:

{
    "instrument" : "EUR_USD",
    "granularity" : "D",
    "candles" : [
        {
            "time" : "2014-07-02T04:00:00.000000Z", // time in RFC3339 format
            "openMid" : 1.36803,
            "highMid" : 1.368125,
            "lowMid" : 1.364275,
            "closeMid" : 1.365315,
            "volume" : 28242,
            "complete" : true
        },
        {
            "time" : "2014-07-03T04:00:00.000000Z", // time in RFC3339 format
            "openMid" : 1.36532,
            "highMid" : 1.366445,
            "lowMid" : 1.35963,
            "closeMid" : 1.3613,
            "volume" : 30487,
            "complete" : false
        }
    ]
}

My problem is that when I load it using Pandas, instrument, granularity, and candles are processed as the column titles. However, I want to use time, openMid, highMid, lowMid, closeMid, volume, and complete to create my columns. But they are just processed as a belonging to candles. Any ideas on how I can accomplish this? Thanks

Answer 1

You'll have to read the string using the json library first:

import json
data = json.loads(string)

And then you can extract the candles data from the resulting dictionary and build your DataFrame that way, eg:

candles_data = data.pop('candles')
df = pd.DataFrame(candles_data)
for k, v in data.iteritems():
    df[k] = v