Facing issue with parsing the following JSON String in Python

Question

{"tradedDate":"29NOV2019","data":[{"pricebandupper":"77.05","symbol":"YESBANK","applicableMargin":"43.35","bcEndDate":"12-JUN-19","totalSellQuantity":"-","adhocMargin":"9.71","companyName":"Yes Bank Limited","marketType":"N","exDate":"03-JUN-19","bcStartDate":"06-JUN-19","css_status_desc":"Listed","dayHigh":"74.00","basePrice":"70.05","securityVar":"23.26","pricebandlower":"63.05","sellQuantity5":"-","sellQuantity4":"-","sellQuantity3":"-","cm_adj_high_dt":"03-APR-19","sellQuantity2":"-","dayLow":"67.55","sellQuantity1":"-","quantityTraded":"42,83,32,393","pChange":"-2.21","totalTradedValue":"3,03,130.83","deliveryToTradedQuantity":"18.95","totalBuyQuantity":"13,12,136","averagePrice":"70.77","indexVar":"-","cm_ffm":"14,283.27","purpose":"ANNUAL GENERAL MEETING\/ DIVIDEND - RS 2 PER SHARE","buyPrice2":"-","secDate":"29-Nov-2019 00:00:00","buyPrice1":"68.30","high52":"286.00","previousClose":"70.05","ndEndDate":"-","low52":"29.00","buyPrice4":"-","buyPrice3":"-","recordDate":"-","deliveryQuantity":"8,11,80,641","buyPrice5":"-","priceBand":"No Band","extremeLossMargin":"10.38","cm_adj_low_dt":"01-OCT-19","varMargin":"23.26","sellPrice1":"-","sellPrice2":"-","totalTradedVolume":"42,83,32,393","sellPrice3":"-","sellPrice4":"-","sellPrice5":"-","change":"-1.55","surv_indicator":"-","ndStartDate":"-","buyQuantity4":"-","isExDateFlag":false,"buyQuantity3":"-","buyQuantity2":"-","buyQuantity1":"13,12,136","series":"EQ","faceValue":"2.00","buyQuantity5":"-","closePrice":"68.30","open":"72.00","isinCode":"INE528G01027","lastPrice":"68.50"}],"optLink":"\/marketinfo\/sym_map\/symbolMapping.jsp?symbol=YESBANK&instrument=-&date=-&segmentLink=17&symbolCount=2","otherSeries":["EQ"],"futLink":"\/live_market\/dynaContent\/live_watch\/get_quote\/GetQuoteFO.jsp?underlying=YESBANK&instrument=FUTSTK&expiry=26DEC2019&type=-&strike=-","lastUpdateTime":"29-NOV-2019 15:59:59"}

This is the JSON string I am fetching from a website, I am trying to parse it using the Json.load as well as Json.load but get the following error.

stock_data = get_hl(soup)
File "/home/shreel/oi-bot/oi.py", line 349, in get_hl
final_op = json.loads(json_str)
File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.7/json/decoder.py", line 380, in raw_decode
obj, end = self.scan_once(s, idx)
ValueError: Expecting object: line 1 column 1555 (char 1554)

My code is as follow:

    final_op = json.loads(json_str)
    print(final_op)

The variable json_str has that string I posted above.

Can anyone help me out understanding the error as well as how to go about to solve it?

PS Not from Python Programming Background.

Answer 1

Had to replace "false" with "False" to be able to use the given input in the example below

Based on the comments and the question, to get DayHigh and DayLow you must address this json object as a dictionary that contains nested lists and dictionaries. You must find where DayHigh and DayLow are located and then you can simply address them as you would in a dictionary with dictionaries and lists inside it. Here is an example:

json_data = {"tradedDate":"29NOV2019","data":[{"pricebandupper":"77.05","symbol":"YESBANK","applicableMargin":"43.35","bcEndDate":"12-JUN-19","totalSellQuantity":"-","adhocMargin":"9.71","companyName":"Yes Bank Limited","marketType":"N","exDate":"03-JUN-19","bcStartDate":"06-JUN-19","css_status_desc":"Listed","dayHigh":"74.00","basePrice":"70.05","securityVar":"23.26","pricebandlower":"63.05","sellQuantity5":"-","sellQuantity4":"-","sellQuantity3":"-","cm_adj_high_dt":"03-APR-19","sellQuantity2":"-","dayLow":"67.55","sellQuantity1":"-","quantityTraded":"42,83,32,393","pChange":"-2.21","totalTradedValue":"3,03,130.83","deliveryToTradedQuantity":"18.95","totalBuyQuantity":"13,12,136","averagePrice":"70.77","indexVar":"-","cm_ffm":"14,283.27","purpose":"ANNUAL GENERAL MEETING/ DIVIDEND - RS 2 PER SHARE","buyPrice2":"-","secDate":"29-Nov-2019 00:00:00","buyPrice1":"68.30","high52":"286.00","previousClose":"70.05","ndEndDate":"-","low52":"29.00","buyPrice4":"-","buyPrice3":"-","recordDate":"-","deliveryQuantity":"8,11,80,641","buyPrice5":"-","priceBand":"No Band","extremeLossMargin":"10.38","cm_adj_low_dt":"01-OCT-19","varMargin":"23.26","sellPrice1":"-","sellPrice2":"-","totalTradedVolume":"42,83,32,393","sellPrice3":"-","sellPrice4":"-","sellPrice5":"-","change":"-1.55","surv_indicator":"-","ndStartDate":"-","buyQuantity4":"-","isExDateFlag":False,"buyQuantity3":"-","buyQuantity2":"-","buyQuantity1":"13,12,136","series":"EQ","faceValue":"2.00","buyQuantity5":"-","closePrice":"68.30","open":"72.00","isinCode":"INE528G01027","lastPrice":"68.50"}],"optLink":"/marketinfo/sym_map/symbolMapping.jsp?symbol=YESBANK&instrument=-&date=-&segmentLink=17&symbolCount=2","otherSeries":["EQ"],"futLink":"/live_market/dynaContent/live_watch/get_quote/GetQuoteFO.jsp?underlying=YESBANK&instrument=FUTSTK&expiry=26DEC2019&type=-&strike=-","lastUpdateTime":"29-NOV-2019 15:59:59"}

for keys in range(len(json_data['data'])):
    print(json_data['data'][keys]['dayLow'])
    print(json_data['data'][keys]['dayHigh'])

Output:

67.55
74.00

Also if there are multiple json objects/files you can place them in a loop and iterate through it. If you want to turn it into a table then you can create a dictionary and make a dataframe out of it:

data_to_df = {'dayLow':[],'dayHigh':[]}
for keys in range(len(json_data['data'])):
    data_to_df['dayLow'].append(json_data['data'][keys]['dayLow'])
    data_to_df['dayHigh'].append(json_data['data'][keys]['dayHigh'])
import pandas as pd
df = pd.DataFrame(data_to_df)
print(df)

Output:

  dayLow dayHigh
0  67.55   74.00

Answer 2

According to JSON standards, boolean values need to be lowercased (false / true). Read more here

The API that you are using, doesn't abide by this, and hence it's causing the above error. If changing the API return value is not an option, a quick hotfix would be:

import re
json_str = re.sub(json_str, False, false)
final_ob = json.loads(json_str)