优雅的方式来获得价值Boto3响应字典

Question

I'm using boto3 to describe my pipelines to me. It throws back a response in a very curious format -

{'ResponseMetadata': {'HTTPStatusCode': 200,
                      'RequestId': '2c42a320-c6ab-11e5-a021-afa8e3e7ca24'},
 u'pipelineDescriptionList': [{u'fields': [{u'key': u'key1', u'stringValue': u'AIDAIxxxxxxNWNI6I'},
                                           {u'key': u'key2', u'stringValue': u'2016-01-29T13:32:09'},
                                           {u'key': u'key3', u'stringValue': u'ABCD'},
                                           {u'key': u'key4', u'stringValue': u'PIPELINE'}],
                               u'name': u'ABCD',
                               u'pipelineId': u'df-03442XXXXAZTRKE4NNK',
                               u'tags': []},
                              {u'fields': [{u'key': u'key1',u'stringValue': u'2016-01-29T11:26:02'},
                                           {u'key': u'key2', u'stringValue': u'[{"key":"jobrun","value":"RzIN"},{"key":"env","value":"prod"}]'},
                                           {u'key': u'key3', u'stringValue': u'2016-01-29T11:25:20'},
                                           {u'key': u'key4', u'stringValue': u'PIPELINE'}],
                               u'name': u'test-pipeline',
                               u'pipelineId': u'df-006474936ZXRQ238Q70O',
                               u'tags': [{u'key': u'jobrun', u'value': u'RWssN'},{u'key': u'env', u'value': u'prod'}]}]}

Now I'm interested in getting pipeline 'ABCD's key2 value. If I try to get it in the most intuitive way, it gets very messy since I'll have to iterate through each of the pipelineDescriptionList list items, iterate through each of the list items in 'fields', check the one for which key equals key2 , and get the stringValue for that.

My question is, are we actually expected to deal with this mess? Or is there some Python magic that can make my task more elegant?

Answer 1

One option would be to use jmespath which is bundled with boto3. It's a pretty powerful tool for processing JSON-like data structures.

For example, if the response you show above was bound to the variable response you could do this:

jmespath.search("pipelineDescriptionList[?name == 'ABCD'].fields[] | [?key == 'key2'].stringValue", response)

which would return:

[u'2016-01-29T13:32:09']

There may be better ways to do it, that's just a quick cut at it. You can also compile these expressions to make things a bit more efficient for doing many searches with the same query.