How to decode JavaScript response in python 3.4+?

Question

I got this response from a website. I really don't know how to decode it and get the information that I want.

Here is the JavaScript response:

{header:{code:0,message:{title:"",detail:""}},body:{parameters:{},dataStores:{"kccjStore":{pageSize:100,pageNumber:1,recordCount:3,name:"kccjStore",rowSetName:"pojo_com.neusoft.education.sysu.xscj.xscjcx.model.KccjModel",order:"kclb asc",metaData:{"jxbmc":{dataType:12},"njzypm":{dataType:12},"xh":{dataType:12},"jxbh":{dataType:12},"sftg":{dataType:12},"zzcj":{dataType:12},"cjzt":{dataType:12},"bzw":{dataType:12},"class":{dataType:12},"xs":{dataType:12},"xq":{dataType:12},"jsxm":{dataType:12},"xnd":{dataType:12},"resource_id":{dataType:12},"jxbpm":{dataType:12},"jd":{dataType:12},"kch":{dataType:12},"kcywmc":{dataType:12},"kcmc":{dataType:12},"khfs":{dataType:12},"cjlcId":{dataType:12},"xf":{dataType:12},"kclb":{dataType:12},"zpcj":{dataType:12}},rowSet:{primary:[{"jxbpm":"79\/93","bzw":"00","cjzt":"2","cjlcId":"11895237815","kcywmc":"Mobile Information Engineering Training:Intermediate ","jxbh":"46000092161002","zpcj":"90","resource_id":"11895237815","xnd":"2016-2017","xq":"1","kch":"46000092","kcmc":"移动信息工程实训（中级）","kclb":"11","xf":"3.0","xs":"72.0","zzcj":"90","jd":"4.0","sftg":"1","xh":"14353379","njzypm":"304\/403","class":"class com.neusoft.education.sysu.xscj.xscjcx.model.KccjModel"},{"jxbpm":"74\/128","bzw":"00","cjzt":"2","cjlcId":"12825792491","kcywmc":"Database Systems Lab","jxbh":"46000085161003","jsxm":"桑应朋","khfs":"03","zpcj":"94","resource_id":"12825792491","xnd":"2016-2017","xq":"1","kch":"46000085","kcmc":"数据库系统实验","kclb":"21","xf":"1.0","xs":"36.0","zzcj":"94","jd":"4.4","sftg":"1","xh":"14353379","njzypm":"157\/403","class":"class com.neusoft.education.sysu.xscj.xscjcx.model.KccjModel"},{"jxbpm":"20\/167","bzw":"00","cjzt":"2","cjlcId":"12825622928","kcywmc":"Database Systems","jxbh":"46000084161001","jsxm":"桑应朋","khfs":"01","zpcj":"87","resource_id":"12825622928","xnd":"2016-2017","xq":"1","kch":"46000084","kcmc":"数据库系统","kclb":"21","xf":"2.0","xs":"36.0","zzcj":"87","jd":"3.7","sftg":"1","xh":"14353379","njzypm":"34\/403","class":"class com.neusoft.education.sysu.xscj.xscjcx.model.KccjModel"}]}}}}}

I am working with Python 3.4. Below is my code

html_doc = opener.open('http://wjw.sysu.edu.cn/api/score?year=2016-2017&term=1&pylb=01').read().decode('utf-8')

Thanks in advance!

——————————————————

I mistakenly thought it was a JSON file before and asked a bad question. I'm sorry for that. But thank you for helping me! Hope some of you can help me with my new question!

Answer 1

There are a couple of problems with your input:

1. The braces on your input are unbalanced. You're missing the opening brace, { .
2. JSON requires that all keys be quoted using double quotes. Your input uses a different syntax, which is valid Javascript but not valid JSON. You can parse such objects into python dictionaries, if you really have to, using a third party package such as demjson .

>>> import demjson
>>> parsed = demjson.decode('{header:{code:0,message:{title:"",detail:""}}}')
>>> parsed
{u'header': {u'message': {u'detail': '', u'title': ''}, u'code': 0}}

That being said, your original instinct to use json.loads was generally correct. If you have any control over your input whatsoever (eg if it's the output of an API or script that you can re-write,) you should re-write it to output strict JSON , making it parsable by json.loads .

Answer 2

It looks like you're data is not strictly JSON because some of the keys are not strings.

Also, there seems to be a missing matching curly brace -- should add as the first character,

{header:{code:0,message:{title:"",detail:""}},body:{parameters:{},dataStores:{"kccjStore":{pageSize:100,pageNumber:1,recordCount:3,name:"kccjStore",rowSetName:"pojo_com.neusoft.education.sysu.xscj.xscjcx.model.KccjModel",order:"kclb asc",metaData:{"jxbmc":{dataType:12},"njzypm":{dataType:12},"xh":{dataType:12},"jxbh":{dataType:12},"sftg":{dataType:12},"zzcj":{dataType:12},"cjzt":{dataType:12},"bzw":{dataType:12},"class":{dataType:12},"xs":{dataType:12},"xq":{dataType:12},"jsxm":{dataType:12},"xnd":{dataType:12},"resource_id":{dataType:12},"jxbpm":{dataType:12},"jd":{dataType:12},"kch":{dataType:12},"kcywmc":{dataType:12},"kcmc":{dataType:12},"khfs":{dataType:12},"cjlcId":{dataType:12},"xf":{dataType:12},"kclb":{dataType:12},"zpcj":{dataType:12}},rowSet:{primary:[{"jxbpm":"79\/93","bzw":"00","cjzt":"2","cjlcId":"11895237815","kcywmc":"Mobile Information Engineering Training:Intermediate ","jxbh":"46000092161002","zpcj":"90","resource_id":"11895237815","xnd":"2016-2017","xq":"1","kch":"46000092","kcmc":"移动信息工程实训（中级）","kclb":"11","xf":"3.0","xs":"72.0","zzcj":"90","jd":"4.0","sftg":"1","xh":"14353379","njzypm":"304\/403","class":"class com.neusoft.education.sysu.xscj.xscjcx.model.KccjModel"},{"jxbpm":"74\/128","bzw":"00","cjzt":"2","cjlcId":"12825792491","kcywmc":"Database Systems Lab","jxbh":"46000085161003","jsxm":"桑应朋","khfs":"03","zpcj":"94","resource_id":"12825792491","xnd":"2016-2017","xq":"1","kch":"46000085","kcmc":"数据库系统实验","kclb":"21","xf":"1.0","xs":"36.0","zzcj":"94","jd":"4.4","sftg":"1","xh":"14353379","njzypm":"157\/403","class":"class com.neusoft.education.sysu.xscj.xscjcx.model.KccjModel"},{"jxbpm":"20\/167","bzw":"00","cjzt":"2","cjlcId":"12825622928","kcywmc":"Database Systems","jxbh":"46000084161001","jsxm":"桑应朋","khfs":"01","zpcj":"87","resource_id":"12825622928","xnd":"2016-2017","xq":"1","kch":"46000084","kcmc":"数据库系统","kclb":"21","xf":"2.0","xs":"36.0","zzcj":"87","jd":"3.7","sftg":"1","xh":"14353379","njzypm":"34\/403","class":"class com.neusoft.education.sysu.xscj.xscjcx.model.KccjModel"}]}}}}}

Here is a link to w3schools` JSON reference.

Here is a JSON formatter/validator.

Answer 3

In case you don't want to use third party library, you can just clean up the data by yourself using regex (eg. adding double quote to remaining properties):

import re
import json

text = # the input string

p = re.compile(r'({|,)([^{}:"]+):')
text = p.sub(r'\1"\2":', '{' + text)
obj = json.loads(text)

Answer 4

It seems you forgot the first curl brace. The code below can help you parse the data to json:

s = <YOUR DATA>
import json
import re

def repl(m):
    return '"%s":'%m.group(1)

s = re.sub(r'(\w+):', repl, s)
o = json.loads(s)
print o