在python中从json.loads（）解析数据

Question

I'm attempting to analyze data from a website. I parsed the HTML to get the json data using json.loads().

data = json.loads(soup.find('script', type='application/ld+json').text)

So now I'm left with data which resembles the following:

data = """
{'aggregateRating': {'reviewCount': 1691, 
                     '@type'      : 'AggregateRating', 
                     'ratingValue': 4.0}, 
 'review': [{'reviewRating' : {'ratingValue': 5}, 
               'datePublished': '2017-10-31', 
               'description'  : "I had a chance to see the Lakers ...", 
               'author'       : 'Andre W.'}]
}
""""

I am interested in returning the 'ratingValue' integer from reviewRating in the 'review' array. When I run this script:

pd.DataFrame(data['review'], columns = ['reviewRating'])

I get this:

    reviewRating
0   {'ratingValue': 5}

Instead, I'm looking to get data in the form of:

    ratingValue
0   5

I've attempted various variations such as

pd.DataFrame(data['review'], columns = ['reviewRating']['ratingValue'])
pd.DataFrame(data['review'], columns = ['reviewRating'][['ratingValue']])
pd.DataFrame(data['review']['reviewRating'], columns = ['ratingValue'])

But I'm sure i don't understand the underlaying structure of the data, or pandas.

Thus, am I better off cleaning {'ratingValue': 5} as a string in order to be left with the integer of interest, or is there an easy way to create a DataFrame that has the integer value of 'ratingValue'?

Thanks.

Answer 1

If you use json_normalize from pandas.io.json you can create the dataframe directly from the json.

Using your sample data, I was able to output:

>>> frame = json_normalize(data)

     author datePublished                           description  \
0  Andre W.    2017-10-31  I had a chance to see the Lakers ...

   reviewRating.ratingValue
0                         5

And then you can access the rating value using:

frame.at[0, 'reviewRating.ratingValue'] # which should give you 5