来自request.get的无效JSON数据

Question

I am querying a whole-house power monitor (Neurio) that returns data in JSON format.

When I enter the URL in a Chrome browser, http://192.168.1.87/current-sample , I get properly formatted JSON data as follows:

{"sensorId":"0x0000C47F51019B7D","timestamp":"2016-12-24T14:56:08Z","channels":[{"type":"PHASE_A_CONSUMPTION","ch":1,"eImp_Ws":55552784178,"eExp_Ws":23,"p_W":3188,"q_VAR":321,"v_V":121.753},{"type":"PHASE_B_CONSUMPTION","ch":2,"eImp_Ws":62493402411,"eExp_Ws":23,"p_W":3499,"q_VAR":263,"v_V":120.334},{"type":"CONSUMPTION","ch":3,"eImp_Ws":118046186640,"eExp_Ws":41,"p_W":6687,"q_VAR":584,"v_V":121.044}],"cts":[{"ct":1,"p_W":3188,"q_VAR":321,"v_V":121.753},{"ct":2,"p_W":3499,"q_VAR":263,"v_V":120.334},{"ct":3,"p_W":0,"q_VAR":0,"v_V":0.000},{"ct":4,"p_W":0,"q_VAR":0,"v_V":121.747}]}

which parses correctly in JSONLint.

When I attempt to pull the same data in Python using the following line of code:

pvdata = requests.get('http://'+neurioip+'/current-sample').json()

The returned data includes invalid characters as in the following example. (Data retrieved by simply printing pvdata)

{u'channels': [{u'eExp_Ws': 23, u'v_V': 122.434, u'ch': 1, u'eImp_Ws': 55554346060, u'q_VAR': 305, u'p_W': 1489, u'type': u'PHASE_A_CONSUMPTION'}, {u'eExp_Ws': 23, u'v_V': 120.981, u'ch': 2, u'eImp_Ws': 62495160471, u'q_VAR': 237, u'p_W': 1872, u'type': u'PHASE_B_CONSUMPTION'}, {u'eExp_Ws': 41, u'v_V': 121.708, u'ch': 3, u'eImp_Ws': 118049506582, u'q_VAR': 542, u'p_W': 3360, u'type': u'CONSUMPTION'}], u'sensorId': u'0x0000C47F51019B7D', u'cts': [{u'p_W': 1489, u'q_VAR': 305, u'v_V': 122.434, u'ct': 1}, {u'p_W': 1872, u'q_VAR': 237, u'v_V': 120.981, u'ct': 2}, {u'p_W': 0, u'q_VAR': 0, u'v_V': 0.0, u'ct': 3}, {u'p_W': 0, u'q_VAR': 0, u'v_V': 122.432, u'ct': 4}], u'timestamp': u'2016-12-24T15:03:42Z'}

Data retreived using the following code:

for keys,values in pvdata.items(): print(keys) print(values)

channels [{u'eExp_Ws': 23, u'v_V': 122.843, u'ch': 1, u'eImp_Ws': 55555370977, u'q_VAR': 14, u'p_W': 230, u'type': u'PHASE_A_CONSUMPTION'}, {u'eExp_Ws': 23, u'v_V': 121.088, u'ch': 2, u'eImp_Ws': 62496733790, u'q_VAR': -3, u'p_W': 661, u'type': u'PHASE_B_CONSUMPTION'}, {u'eExp_Ws': 41, u'v_V': 121.965, u'ch': 3, u'eImp_Ws': 118052104817, u'q_VAR': 12, u'p_W': 890, u'type': u'CONSUMPTION'}] sensorId 0x0000C47F51019B7D cts [{u'p_W': 230, u'q_VAR': 14, u'v_V': 122.843, u'ct': 1}, {u'p_W': 661, u'q_VAR': -3, u'v_V': 121.088, u'ct': 2}, {u'p_W': 0, u'q_VAR': 0, u'v_V': 0.0, u'ct': 3}, {u'p_W': 0, u'q_VAR': 0, u'v_V': 122.84, u'ct': 4}] timestamp 2016-12-24T15:25:16Z

The "u" characters makes this unparsable in JSONlint or in subsequent lines of code in my program.

I've looked at default character encoding in my Python environment but that doesn't seem to lead anywhere. I'm looking for other ideas to investigate.

Answer 1

.json() is what parses the JSON. It's done. You have a python dictionary which you can use normally now (as you've already seen by iterating through it).

Answer 2

It is unparsable since the string representation of a dict simply isn't JSON; you need to remarshall the dict to JSON if that's what you need:

In [36]: d = {u'channels': [{u'eExp_Ws': 23, u'v_V': 122.434, u'ch': 1, u'eImp_Ws': 55554346060, u'q_VAR': 305, u'p_W': 1489, u'type': u'PHASE_A_CONSUMPTION'}, {u'eExp_Ws': 23, u'v_V': 120.981, u'ch': 2, u'eImp_Ws': 62495160471, u'q_VAR': 237, u'p_W': 1872, u'type': u'PHASE_B_CONSUMPTION'}, {u'eExp_Ws': 41, u'v_V': 121.708, u'ch': 3, u'eImp_Ws': 118049506582, u'q_VAR': 542, u'p_W': 3360, u'type': u'CONSUMPTION'}], u'sensorId': u'0x0000C47F51019B7D', u'cts': [{u'p_W': 1489, u'q_VAR': 305, u'v_V': 122.434, u'ct': 1}, {u'p_W': 1872, u'q_VAR': 237, u'v_V': 120.981, u'ct': 2}, {u'p_W': 0, u'q_VAR': 0, u'v_V': 0.0, u'ct': 3}, {u'p_W': 0, u'q_VAR': 0, u'v_V': 122.432, u'ct': 4}], u'timestamp': u'2016-12-24T15:03:42Z'}

In [38]: json.dumps(d)
Out[38]: '{"sensorId": "0x0000C47F51019B7D", "timestamp": "2016-12-24T15:03:42Z", "cts": [{"q_VAR": 305, "ct": 1, "v_V": 122.434, "p_W": 1489}, {"q_VAR": 237, "ct": 2, "v_V": 120.981, "p_W": 1872}, {"q_VAR": 0, "ct": 3, "v_V": 0.0, "p_W": 0}, {"q_VAR": 0, "ct": 4, "v_V": 122.432, "p_W": 0}], "channels": [{"q_VAR": 305, "type": "PHASE_A_CONSUMPTION", "p_W": 1489, "v_V": 122.434, "ch": 1, "eImp_Ws": 55554346060, "eExp_Ws": 23}, {"q_VAR": 237, "type": "PHASE_B_CONSUMPTION", "p_W":
1872, "v_V": 120.981, "ch": 2, "eImp_Ws": 62495160471, "eExp_Ws": 23}, {"q_VAR": 542, "type": "CONSUMPTION", "p_W": 3360, "v_V": 121.708, "ch": 3, "eImp_Ws": 118049506582, "eExp_Ws": 41}]}'