Trouble accessing data multiple objects in JSON with Python
Question
I am trying to parse through a web-service and retrieve certain records, however I am consistently receiving a KeyError for responses with more than one object. These records are returned in intervals, so sometimes I might receive one record and others I might receive 300. If I receive one record, the logic of my code works, if there are multiple items then the code doesn't work.
Here is an example of an output with one object.
{
"status": {
"code": 311,
"message": "Service Request Successfully Queried.",
"cause": ""
},
"Response": {
"NumOutputObjects": "1",
"ListOfServiceRequest": {
"ServiceRequest": [
{
"AddressVerified": "Y",
"SRNumber": "1-13967451",
"SRType": "Service Not Complete",
"CreatedDate": "05/08/2015 10:00:38",
"UpdatedDate": "05/08/2015 10:00:49",
"IntegrationId": "05082015100148678",
"Status": "Open",
"CreatedByUserLogin": "PROXYE",
"UpdatedByUserLogin": "PROXYE",
"Anonymous": "N",
"Zipcode": "90032",
"Latitude": "34.0843242531",
"Longitude": "-118.171015007",
"CustomerAccessNumber": "",
"LADWPAccountNo": "",
"NewContactFirstName": "jj",
"NewContactLastName": "rambo",
"NewContactPhone": "",
"NewContactEmail": "",
"ParentSRNumber": "1-10552271",
"Priority": "Normal",
"Language": "",
"ReasonCode": "",
"ServiceDate": "",
"Source": "",
"ClosedDate": "",
"Email": "",
"FirstName": "",
"HomePhone": "",
"LastName": "",
"LoginUser": "",
"ResolutionCode": "",
"SRUnitNumber": "",
"MobilOS": "",
"SRAddress": "5163 E TEMPLETON ST, 90032",
"SRAddressName": "",
"SRAreaPlanningCommission": "East Los Angeles APC",
"SRCommunityPoliceStation": "CENTRAL BUREAU",
"SRCouncilDistrictMember": "Jose Huizar",
"SRCouncilDistrictNo": "14",
"SRDirection": "E",
"SRNeighborhoodCouncilId": "48",
"SRNeighborhoodCouncilName": "LA-32 NC",
"SRStreetName": "TEMPLETON",
"SRSuffix": "ST",
"SRTBColumn": "F",
"SRTBMapGridPage": "595",
"SRTBRow": "6",
"SRXCoordinate": "6509897",
"SRYCoordinate": "1853117",
"AssignTo": "NC",
"Assignee": "NC eWaste Supervisor 01",
"Owner": "BOS",
"ParentSRStatus": "Open",
"ParentSRType": "Electronic Waste",
"ParentSRLinkDate": "05/08/2015 10:00:39",
"ParentSRLinkUser": "PROXYE",
"SRAreaPlanningCommissionId": "5",
"SRCommunityPoliceStationAPREC": "HOLLENBECK",
"SRCommunityPoliceStationPREC": "4",
"SRCrossStreet": "",
"ActionTaken": "",
"SRCity": "",
"RescheduleCounter": "",
"SRHouseNumber": "5163",
"SourceofRequestCouncil": "",
"CCBPremiseType": "",
"ContainerBlackCount": "",
"ContainerBrownCount": "",
"SRIntersectionDirection": "",
"SRApproximateAddress": "N",
"ContainerGreenCount": "",
"OtherBureauName": "",
"AssigneeName": "",
"AssigneeOrganization": "E-Waste, NC",
"AnotherBureauEmailId": "",
"ListOfLa311BarricadeRemoval": {},
"ListOfLa311BulkyItem": {},
"ListOfLa311DeadAnimalRemoval": {},
"ListOfLa311GraffitiRemoval": {},
"ListOfLa311InformationOnly": {},
"ListOfLa311MultipleStreetlightIssue": {},
"ListOfLa311SingleStreetlightIssue": {},
"ListOfLa311SrPhotoId": {
"La311SrPhotoId": []
},
"ListOfLa311BusPadLanding": {},
"ListOfLa311CurbRepair": {},
"ListOfLa311Flooding": {},
"ListOfLa311GeneralStreetInspection": {},
"ListOfLa311GuardWarningRailMaintenance": {},
"ListOfLa311GutterRepair": {},
"ListOfLa311LandMudSlide": {},
"ListOfLa311Pothole": {},
"ListOfLa311Resurfacing": {},
"ListOfLa311SidewalkRepair": {},
"ListOfLa311StreetSweeping": {},
"ListOfLa311BeesOrBeehive": {},
"ListOfLa311MedianIslandMaintenance": {},
"ListOfLa311OvergrownVegetationPlants": {},
"ListOfLa311PalmFrondsDown": {},
"ListOfLa311StreetTreeInspection": {},
"ListOfLa311StreetTreeViolations": {},
"ListOfLa311TreeEmergency": {},
"ListOfLa311TreeObstruction": {},
"ListOfLa311TreePermits": {},
"ListOfLa311BrushItemsPickup": {},
"ListOfLa311Containers": {},
"ListOfLa311ElectronicWaste": {},
"ListOfLa311IllegalDumpingPickup": {},
"ListOfLa311ManualPickup": {},
"ListOfLa311MetalHouseholdAppliancesPickup": {},
"ListOfLa311MoveInMoveOut": {},
"ListOfLa311HomelessEncampment": {},
"ListOfLa311IllegalAutoRepair": {},
"ListOfLa311IllegalConstruction": {},
"ListOfLa311IllegalConstructionFence": {},
"ListOfLa311IllegalDischargeOfWater": {},
"ListOfLa311IllegalDumpingInProgress": {},
"ListOfLa311IllegalExcavation": {},
"ListOfLa311IllegalSignRemoval": {},
"ListOfLa311IllegalVending": {},
"ListOfLa311LeafBlowerViolation": {},
"ListOfLa311NewsRackViolation": {},
"ListOfLa311Obstructions": {},
"ListOfLa311TablesAndChairsObstructing": {},
"ListOfLa311GisLayer": {
"La311GisLayer": [
{
"A_Call_No": "",
"Area": "0",
"Day": "MONDAY",
"DirectionSuffix": "",
"DistrictAbbr": "",
"DistrictName": "NC",
"DistrictNumber": "",
"DistrictOffice": "",
"Fraction": "",
"R_Call_No": "",
"SectionId": "",
"ShortDay": "",
"StreetFrom": "",
"StreetTo": "",
"StreetLightId": "",
"StreetLightStatus": "",
"Type": "GIS",
"Y_Call_No": "",
"Name": "05082015100148678100",
"CommunityPlanningArea": "",
"LastUpdatedBy": "",
"BOSRadioHolderName": ""
}
]
},
"ListOfLa311ServiceRequestNotes": {
"La311ServiceRequestNotes": [
{
"CreatedDate": "05/08/2015 10:00:39",
"Comment": "Materials have been out in a normal collection area, unsure why driver missed the e-waste items.",
"CreatedByUser": "PROXYE",
"IsSrNoAvailable": "",
"CommentType": "External",
"Notification": "N",
"FeedbackSRType": "",
"IntegrationId": "050820151001486782",
"Date1": "",
"Date2": "",
"Date3": "",
"Text1": "",
"AnotherBureau": "",
"EmailAddress": "",
"ListOfLa311SrNotesAuditTrail": {}
},
{
"CreatedDate": "05/08/2015 10:00:39",
"Comment": "",
"CreatedByUser": "PROXYE",
"IsSrNoAvailable": "",
"CommentType": "Address Comments",
"Notification": "N",
"FeedbackSRType": "",
"IntegrationId": "050820151001486781",
"Date1": "",
"Date2": "",
"Date3": "",
"Text1": "",
"AnotherBureau": "",
"EmailAddress": "",
"ListOfLa311SrNotesAuditTrail": {}
}
]
},
"ListOfLa311SubscribeDuplicateSr": {},
"ListOfChildServiceRequest": {},
"ListOfLa311BillingCsscAdjustment": {},
"ListOfLa311BillingEccAdjustment": {},
"ListOfLa311BillingRsscAdjustment": {},
"ListOfLa311BillingRsscExemption": {},
"ListOfLa311SanitationBillingBif": {},
"ListOfLa311SanitationBillingCssc": {},
"ListOfLa311SanitationBillingEcc": {},
"ListOfLa311SanitationBillingLifeline": {},
"ListOfLa311SanitationBillingRssc": {},
"ListOfLa311SanitationBillingSrf": {},
"ListOfLa311DocumentLog": {},
"ListOfAuditTrailItem2": {
"AuditTrailItem2": [
{
"Date": "05/08/2015 10:00:49",
"EmployeeLogin": "SADMIN",
"Field": "Assignee",
"NewValue": "NC eWaste Supervisor 01",
"OldValue": ""
}
]
},
"ListOfLa311GenericBc": {
"La311GenericBc": [
{
"ATTRIB_08": "",
"NAME": "05082015100148678100",
"PAR_ROW_ID": "1-8BDCR",
"ROW_ID": "1-8BOCG",
"TYPE": "GIS",
"ATTRIB_16": "",
"ListOfLa311GenericbcAuditTrail": {}
},
{
"ATTRIB_08": "",
"NAME": "05082015100148678",
"PAR_ROW_ID": "1-8BDCR",
"ROW_ID": "1-8BOCJ",
"TYPE": "Service Not Complete",
"ATTRIB_16": "",
"ListOfLa311GenericbcAuditTrail": {}
}
]
},
"ListOfLa311ServiceNotComplete": {
"La311ServiceNotComplete": [
{
"ContainerLocation": "",
"ContainerType": "",
"DriverFirstName": "",
"DriverLastName": "",
"MissedCollectionService": "Electronic Waste",
"OtherServiceMissedReason": "",
"ServiceDateRendered": "",
"ServiceMissedReason": "I'm not sure",
"TruckNo": "",
"Type": "Service Not Complete",
"WireBasketLocation": "",
"LastUpdatedBy": "",
"Name": "05082015100148678"
}
]
},
"ListOfLa311Other": {},
"ListOfLa311WeedAbatementForPrivateParcels": {},
"ListOfLa311SanitationBillingInquiry": {}
}
]
}
}
}
The code is below;
data2 = jsonpickle.decode((f2.read()))
Start = datetime.datetime.now()
data2 = jsonpickle.encode(data2)
url2 = "myURL"
headers2 = {'Content-type': 'text/plain', 'Accept': '/'}
r2 = requests.post(url2, data=data2, headers=headers2)
decoded2 = json.loads(r2.text)
try:
r2
except requests.exceptions.ConnectTimeout as e:
print "Too slow Mojo!"
items = []
for sr in decoded2['Response']['ListOfServiceRequest']['ServiceRequest']:
SRAddress = sr['SRAddress']
Latitude = sr['Latitude']
Longitude = sr['Longitude']
ReasonCode = sr['ReasonCode']
SRNumber = sr['SRNumber']
FirstName = sr['FirstName']
LastName = sr['LastName']
ResolutionCode = sr['ResolutionCode']
HomePhone = sr['HomePhone']
CreatedDate = sr['CreatedDate']
UpdatedDate = sr['UpdatedDate']
CreatedDate = datetime.datetime.strptime(CreatedDate, "%m/%d/%Y %H:%M:%S")
UpdatedDate = datetime.datetime.strptime(UpdatedDate, "%m/%d/%Y %H:%M:%S")
print SRAddress
print SRNumber
ItemInfo = " "
for ew in sr["ListOfLa311ServiceRequestNotes"][u"La311ServiceRequestNotes"]:
Comment = ew['Comment']
print Comment
Output Materials have been out in a normal collection area, unsure why driver missed the e-waste items.
If I use the logic above for a response with more than one object returned I receive a KeyError value and am unable to access the array that I want to parse.
Example of code with multiple objects returned;
Output when I use; I receive a key error if I attempt to do something along the lines of CommodityType = sr['ListOfLa311ElectronicWaste']['ElectronicWasteType'] below
for sr in decoded2['Response']['ListOfServiceRequest']['ServiceRequest']:
CommodityType = sr['ListOfLa311ElectronicWaste']
# ItemType = sr['ElectronicWestType']
# DriverFirstName = sr ['DriverFirstName']
# DriverLastName = sr ['DriverLastName']
# ItemCount = sr['ItemCount']
# ItemInfo += '{0}, {1}, '.format(ItemType, ItemCount)
# ParentNumber = sr['Name']
#
print CommodityType
{u'La311ElectronicWaste': [{u'IllegallyDumped': u'N', u'OtherElectronicWestType': u'hash', u'ItemCount': u'5', u'Name': u'6a31f058-ece1-4e7d-b682-7d9052a512f4', u'MobileHomeSpace': u'', u'DriverLastName': u'', u'ActiveStatus': u'Y', u'DriverFirstName': u'', u'LastUpdatedBy': u'', u'GatedCommunityMultifamilyDwelling': u'Outside the main gate', u'IllegalDumpCollectionLoc': u'', u'ElectronicWestType': u'Other', u'CollectionLocation': u'Gated Community', u'Type': u'Electronic Waste', u'ServiceDateRendered': u'', u'TruckNo': u''}]}
{u'La311ElectronicWaste': [{u'IllegallyDumped': u'Y', u'OtherElectronicWestType': u'', u'ItemCount': u'5', u'Name': u'3f4d9d20-a712-4be3-822f-e6a45219c1cf', u'MobileHomeSpace': u'', u'DriverLastName': u'', u'ActiveStatus': u'Y', u'DriverFirstName': u'', u'LastUpdatedBy': u'', u'GatedCommunityMultifamilyDwelling': u'', u'IllegalDumpCollectionLoc': u'Cul De Sac', u'ElectronicWestType': u'Electronic Equipment', u'CollectionLocation': u'Alley', u'Type': u'Electronic Waste', u'ServiceDateRendered': u'', u'TruckNo': u''}]}
How do I handle multiple outputs the same way that I have handled the single output?
3 answers
solution1
0 2015-05-08 19:55:25
You haven't provided a program that runs at all, but my guess is you're getting a list of objects, and you need to iterate through the list, then index each object the way you do with a single item. If I put two copies of the object you pasted in a list and store it in a file ìnput.json , I need to read each object like the following:
import json
with open("input.json") as f:
data = json.load(f)
for obj in data:
for service_request in obj["Response"]['ListOfServiceRequest']['ServiceRequest']:
print service_request # obviously do something useful instead
solution2
0 2015-05-08 21:31:34
I think a nested loop will do what you require. sr is the outer loop. ew should be an inner loop nested within sr all you do is ident the second loop.
data2 = jsonpickle.decode((f2.read()))
Start = datetime.datetime.now()
data2 = jsonpickle.encode(data2)
url2 = "myURL"
headers2 = {'Content-type': 'text/plain', 'Accept': '/'}
r2 = requests.post(url2, data=data2, headers=headers2)
decoded2 = json.loads(r2.text)
try:
r2
except requests.exceptions.ConnectTimeout as e:
print "Too slow Mojo!"
items = []
for sr in decoded2['Response']['ListOfServiceRequest']['ServiceRequest']:
SRAddress = sr['SRAddress']
Latitude = sr['Latitude']
Longitude = sr['Longitude']
ReasonCode = sr['ReasonCode']
SRNumber = sr['SRNumber']
FirstName = sr['FirstName']
LastName = sr['LastName']
ResolutionCode = sr['ResolutionCode']
HomePhone = sr['HomePhone']
CreatedDate = sr['CreatedDate']
UpdatedDate = sr['UpdatedDate']
CreatedDate = datetime.datetime.strptime(CreatedDate, "%m/%d/%Y %H:%M:%S")
UpdatedDate = datetime.datetime.strptime(UpdatedDate, "%m/%d/%Y %H:%M:%S")
print SRAddress
print SRNumber
ItemInfo = " "
for ew in sr["ListOfLa311ServiceRequestNotes"][u"La311ServiceRequestNotes"]:
Comment = ew['Comment']
print Comment
for ew in sr["ListOfLa311ElectronicWaste"][u"La311ElectronicWaste"]:
IllegallyDumped = ew['IllegallyDumped']
print IllegallyDumped
I think issue is that ListOfLa311ElectronicWaste is another list - so you need to iterate through this also to access its elements
solution3
0 2015-05-10 00:39:35
I solved this, this thread was a huge reference.
Decoding nested JSON with multiple 'for' loops
for sr in ElectronicType: for illegaldump in ElectronicType['La311ElectronicWaste']: illegalewaste = illegaldump['IllegallyDumped']
Y
N
N
N
N
N
Y
Y
N
Y
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.
Related Question
Trouble accessing nested JSON data in Python
Trouble accessing JSON data in Python for loop
Trouble accessing data from a JSON object that was originally a Python dictionary in Flask
TypeError : Trouble accessing JSON metadata with Python
trouble accessing an objects properties
Trouble deleting certain nested JSON objects in python
Accessing JSON column after reading multiple objects
Python - accessing an element from a JSON objects list
Python accessing data in JSON object
Accessing Nested JSON data in Python
Related Tags