Removing blank lxml element from ElementTree in python

Question

I've been struggling with this for a couple of days now, and I figured I would ask here.

I am working on preparing an XML payload to POST to an Oracle endpoint that contains financials data. I've got most of the XML structured per Oracle specs, but I am struggling with one aspect of it. This is data that will feed the general ledger financial system and the xml structure is below (some elements have been omitted to cut down on the post.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:typ="http://xmlns.oracle.com/apps/financials/generalLedger/journals/desktopEntry/journalImportService/types/" xmlns:jour="http://xmlns.oracle.com/apps/financials/generalLedger/journals/desktopEntry/journalImportService/">
  <soapenv:Header/>
  <soapenv:Body>
    <typ:importJournals>
      <typ:interfaceRows>
        <jour:BatchName>batch</jour:BatchName>
        <jour:AccountingPeriodName>Aug-20</jour:AccountingPeriodName>
        <jour:AccountingDate>2020-08-31</jour:AccountingDate>
        <jour:GlInterface>
          <jour:LedgerId>1234567890</jour:LedgerId>
          <jour:PeriodName>Aug-20</jour:PeriodName>
          <jour:AccountingDate>2020-08-31</jour:AccountingDate>
          <jour:Segment1>1</jour:Segment1>
          <jour:Segment2>1</jour:Segment2>
          <jour:Segment3>1</jour:Segment3>
          <jour:Segment4>1</jour:Segment4>
          <jour:Segment5>0</jour:Segment5>
          <jour:Segment6>0</jour:Segment6>
          <jour:CurrencyCode>USD</jour:CurrencyCode>
          <jour:EnteredCrAmount currencyCode="USD">10.0000</jour:EnteredCrAmount>
        </jour:GlInterface>
        <jour:GlInterface>
          <jour:LedgerId>1234567890</jour:LedgerId>
          <jour:PeriodName>Aug-20</jour:PeriodName>
          <jour:AccountingDate>2020-08-31</jour:AccountingDate>
          <jour:Segment1>2</jour:Segment1>
          <jour:Segment2>2</jour:Segment2>
          <jour:Segment3>2</jour:Segment3>
          <jour:Segment4>2</jour:Segment4>
          <jour:Segment5>0</jour:Segment5>
          <jour:Segment6>0</jour:Segment6>
          <jour:CurrencyCode>USD</jour:CurrencyCode>
          <jour:EnteredDrAmount currencyCode="USD">10.0000</jour:EnteredCrAmount>
        </jour:GlInterface>
      </typ:interfaceRows>
    </typ:importJournals>
  </soapenv:Body>
</soapenv:Envelope>

So if you look at the XML above, within the GlInterface tags, there are 2 per transaction (one is a debit and one is a credit, if you look at the Segments (account codes) they are different, and one GlInterface tag as a EnteredDrAmount tag, while the other has EnteredCrAmount tag.

In the source data, either the Cr or Dr tag is null depending on if the line is a debit or credit, which comes in as "None" in python.

The way I got this to work is to call two calls to get data, one where Cr is not null and one where Dr is not null, and this process works fine, but in Python, I get an error "only one * allowed". Code is below.

    xmlOracle = x_Envelope(
        x_Header,
        x_Body(
            x_importJournals(
                x_interfaceRows(
                    x_h_BatchName(str(batch[0])),
                    x_h_AccountingPeriodName(str(batch[3])),
                    x_h_AccountingDate(str(batch[4])),
                    *[x_GlInterface(
                        x_d_LedgerId(str(adid[0])),
                        x_d_PeriodName(str(adid[1])),
                        x_d_AccountingDate(str(adid[2])),
                        x_d_Segment1(str(adid[5])),
                        x_d_Segment2(str(adid[6])),
                        x_d_Segment3(str(adid[7])),
                        x_d_Segment4(str(adid[8])),
                        x_d_Segment5(str(adid[9])),
                        x_d_Segment6(str(adid[10])),
                        x_d_CurrencyCode(str(adid[11])),
                        x_d_EnteredCrAmount(str(adid[14]), currencyCode=str(adid[11]))
                    ) for adid in CrAdidToProcess],
                    *[x_GlInterface(
                        x_d_LedgerId(str(adid[0])),
                        x_d_PeriodName(str(adid[1])),
                        x_d_AccountingDate(str(adid[2])),
                        x_d_Segment1(str(adid[5])),
                        x_d_Segment2(str(adid[6])),
                        x_d_Segment3(str(adid[7])),
                        x_d_Segment4(str(adid[8])),
                        x_d_Segment5(str(adid[9])),
                        x_d_Segment6(str(adid[10])),
                        x_d_CurrencyCode(str(adid[11])),
                        x_d_EnteredDrAmount(str(adid[14]), currencyCode=str(adid[11]))
                    ) for adid in DrAdidToProcess]
                )
            )
        )
    )

I've also tried making a single call to get the line details and then either removing or filtering out the tag (either Cr or Dr) if it's "None" but I had no luck with this.

While the above process works, there is an error in my code, and I'd like to not have an error in my code.

Thank you all.

Answer 1

After further testing, I believe I figured out the solution to this. I believe I was trying to remove an element from ElementTree object and it was not having any of that. When I passed an element to the remove method/function, it finally worked.

Here is code for the function to remove the "None" entries.

def removeCrDrEmptyElements(element):
    for removeElement in element.xpath('/soapenv:Envelope/soapenv:Body/typ:importJournals/typ:interfaceRows/jour:GlInterface/jour:EnteredCrAmount',
                                         namespaces = { 'soapenv' : 'http://schemas.xmlsoap.org/soap/envelope/',
                                                        'typ' : 'http://xmlns.oracle.com/apps/financials/generalLedger/journals/desktopEntry/journalImportService/types/',
                                                        'jour' : 'http://xmlns.oracle.com/apps/financials/generalLedger/journals/desktopEntry/journalImportService/'
                                                      }):
        if removeElement.text == 'None':
            removeElement.getparent().remove(removeElement)

    for removeElement in element.xpath('/soapenv:Envelope/soapenv:Body/typ:importJournals/typ:interfaceRows/jour:GlInterface/jour:EnteredDrAmount',
                                         namespaces = { 'soapenv' : 'http://schemas.xmlsoap.org/soap/envelope/',
                                                        'typ' : 'http://xmlns.oracle.com/apps/financials/generalLedger/journals/desktopEntry/journalImportService/types/',
                                                        'jour' : 'http://xmlns.oracle.com/apps/financials/generalLedger/journals/desktopEntry/journalImportService/'
                                                      }):
        if removeElement.text == 'None':
            removeElement.getparent().remove(removeElement)

    return element

Obviously this can be better rewritten (which I will do) but I only want to check two elements within the GlInterface tag, the EnteredCrAmount and EnteredDrAmount and remove those elements if the text is None.

Then you can call the function by using the code below to return an element type with the removed Nulls/Nones

xmlWithoutNull = removeCrDrEmptyElements(xmlElement)

Output before running function results:

        <jour:GlInterface>
          # omitted elements
          <jour:EnteredCrAmount currencyCode="USD">1.000000</jour:EnteredCrAmount>
          <jour:EnteredDrAmount currencyCode="USD">None</jour:EnteredDrAmount>
          # omitted elements
        </jour:GlInterface>
        <jour:GlInterface>
          # omitted elements
          <jour:EnteredCrAmount currencyCode="USD">None</jour:EnteredCrAmount>
          <jour:EnteredDrAmount currencyCode="USD">1.000000</jour:EnteredDrAmount>
          # omitted elements
        </jour:GlInterface>

Output after running function results:

        <jour:GlInterface>
          # omitted elements
          <jour:EnteredCrAmount currencyCode="USD">1.000000</jour:EnteredCrAmount>
          # omitted elements
        </jour:GlInterface>
        <jour:GlInterface>
          # omitted elements
          <jour:EnteredDrAmount currencyCode="USD">1.000000</jour:EnteredDrAmount>
          # omitted elements
        </jour:GlInterface>