Python Boto3-数据未正确写入DynamoDB

Question

I have an XML file I am parsing strings from with Python and writing to a DynamoDB table in AWS. The tags are <IMAGE_ID> and <CVSS_FINAL> . When I loop through and print() these values, it returns all of them. However, when I write to Dynamo, only one row of data is written. So, I don't understand why print() returns everything, but only one row is written to the datastore.

Code:

import boto3
import lxml
from lxml import etree

def WriteItemToTable():
    s3 = boto3.resource('s3')
    bucket = ‘xxxxxxxxxxxx’
    key = 'vuln_data.xml'
    dynamo = boto3.client('dynamodb')

    obj = s3.Object(‘xxxxxxxxxx’, 'vuln_data.xml')
    body = obj.get()['Body'].read()

    image_id = etree.fromstring(body).findall('HOST_LIST/HOST/EC2_INFO/IMAGE_ID')
    risk_score = etree.fromstring(body).findall('HOST_LIST/HOST/VULN_INFO_LIST/VULN_INFO/CVSS_FINAL')

    for el in image_id:
        i = el.text
        print(i)

    for el in risk_score:
        j = el.text
        print(j)

    response = dynamo.put_item(
    TableName='ExistingAMI',
        Item={
            'AMI_ID': {
                'S': i
             },
            'CVSS_SCORE': {
                'S': j
            },
         }
       )

WriteItemToTable()

XML:

<HOST_LIST>
    <HOST>
      <EC2_INFO>
        <PUBLIC_DNS_NAME><![CDATA[ec2-xxxxxxxxxxx.compute-1.amazonaws.com]]></PUBLIC_DNS_NAME>
        <IMAGE_ID><![CDATA[ami-xxxxxx]]></IMAGE_ID>
      </EC2_INFO>
      <OPERATING_SYSTEM><![CDATA[Linux x.y]]></OPERATING_SYSTEM>
      <VULN_INFO_LIST>
        <VULN_INFO>
          <QID id="qid_x”>x</QID>
          <TYPE>Vuln</TYPE>
          <CVSS_FINAL>3.5</CVSS_FINAL>
          <RESULT><![CDATA[TLSv1.0 is supported]]></RESULT>
        </VULN_INFO>
        <VULN_INFO>
      <QID id="qid_xxxx">xxxxx</QID>
      <CVSS_FINAL>2.1</CVSS_FINAL>
    </VULN_INFO>
    <VULN_INFO>
      <QID id="qid_xxxx">xxxx</QID>
      <CVSS_FINAL>4.3</CVSS_FINAL>
      <RESULT><![CDATA[TLSv1.0 is supported]]></RESULT>
    </VULN_INFO>
    </VULN_INFO_LIST>
    </HOST>
    <HOST>
      <EC2_INFO>
        <PUBLIC_DNS_NAME><![CDATA[ec2-xxxxxxxxx.compute-1.amazonaws.com]]></PUBLIC_DNS_NAME>
        <IMAGE_ID><![CDATA[ami-yyyyyy]]></IMAGE_ID>
      </EC2_INFO>
      <OPERATING_SYSTEM><![CDATA[Amazon Linux]]></OPERATING_SYSTEM>
      <VULN_INFO_LIST>
        <VULN_INFO>
          <QID id=“x”>x</QID>
          <CVSS_FINAL>3.6</CVSS_FINAL>
        </VULN_INFO>
    </VULN_INFO_LIST>
    </HOST>
</HOST_LIST>

print() output:

ami-xxxxxx
ami-yyyyyy
3.5
3.6

Dynamo Table:

Dynamo Table

Answer 1

While I know nothing of DynamoDB, your Python code should only pass one i and j value as your dynamo.put_item block of code is not nested in either of the for loops and hence takes their last assigned values.

Simply run your image_id and risk_score search together in one loop which can be nested at the <HOST> level. And consider xpath() , available in lxml . And no need for the import lxml call as you import its method etree instead.

doc = etree.fromstring(body)  # PARSE ONLY ONCE

hosts = doc.xpath('//HOST')

for h in hosts:
    i = h.xpath('EC2_INFO/IMAGE_ID')[0].text
    print(i)

    j = h.xpath('VULN_INFO_LIST/VULN_INFO/CVSS_FINAL')[0].text
    print(j)

    response = dynamo.put_item(
        TableName='ExistingAMI',
        Item={
            'AMI_ID': {
                 'S': i
             },
            'CVSS_SCORE': {
                'S': j
            },
         }
    )

WriteItemToTable()

# ami-xxxxxx
# 3.5
# ami-yyyyyy
# 3.6

---

For multiple CVSS_FINAL , use XPath to parse down to the <CSVSS_FINAL> then retrieve corresponding IMAGE_ID with ancestor::*

cvss = obj.xpath('//CVSS_FINAL')   # ALL CVSS_FINAL NODES

for c in cvss:        
    i = c.xpath('ancestor::HOST/EC2_INFO/IMAGE_ID')[0].text
    print(i)

    j = c.text
    print(j)

    k = c.xpath('concat(following-sibling::RESULT, "")')
    print(k)

    response = dynamo.put_item( ... )

# ami-xxxxxx
# 3.5
# TLSv1.0 is supported
# ami-xxxxxx
# 2.1
# 
# ami-xxxxxx
# 4.3
# TLSv1.0 is supported
# ami-yyyyyy
# 3.6
#