Remove "ds:" in signed XML tree namespace in python's signxml

Question

I am working with Python's lxml and signxml to generate an xml file and sign it with a pem certificate and private key.

I am required to validate the signed xml in the followign website validate XML . For some reason in this website the signed XML files with the "ds" namespace in signature tags do not recognize the file as signed.

I will not focus much on the generated xml file with lxml. The code to sign the xml file has the following form:

    def _get_xml_tree_root(self):
        root = ET.Element('facturaElectronicaCompraVenta' , attrib={location_attribute: invoice_sector + '.xsd'})
        xml_header = ET.SubElement(root, 'header')
        xml_detail = ET.SubElement(root, 'detail')
      return root

    def _get_signed_xml(self):
        signed_root = XMLSigner().sign(
            self._get_xml_tree_root(),
            key=base64.b64decode(io.TextIOWrapper(BytesIO(electronic_key)).read()),
            cert=base64.b64decode(io.TextIOWrapper(BytesIO(electronic_certificate)).read())
        )
        return signed_root

The problem is that the xml file that I generate in the signature section has following form:

<facturaElectronicaCompraVenta xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="facturaElectronicaCompraVenta.xsd">
   <header></header>
   <detail></detail>
    <ds:Signature xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
        <ds:SignedInfo>
            <ds:CanonicalizationMethod Algorithm="http://www.w3.org/2006/12/xml-c14n11"/>
            <ds:SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256"/>
            <ds:Reference URI="">
                <ds:Transforms>
                    <ds:Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/>
                    <ds:Transform Algorithm="http://www.w3.org/2006/12/xml-c14n11"/>
                </ds:Transforms>
                <ds:DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"/>
                <ds:DigestValue>KvIMPxajMb98G3+HdSLg1/pgSyisLp4OWZt6Gxhe+/c=</ds:DigestValue>
            </ds:Reference>
        </ds:SignedInfo>
        <ds:SignatureValue>Bv9W9cGyXvX4QeDDb61YME8TbnFlBOVBw2Iiv+a+7VrxjoH4z8kLO4rgonXbqGuk2DKrR4ACqoFQNd/9/lJb31TDk2SjegURBsjP9gLvFWwfq99jh6zn6rPF/gwqd+lA1ruGpDT/Q+vxMXeNpXfk+nDcgdDJoP1bpDEPHbSHGkQu2SX1NQP1SlRZkNoJXxorFfbTDmm1/VFRsv5uBNQvf7hSxTEvvLW8WVYN271iTzHTpAnbyg7VTeys/Ca2FQsZ95hgCHfKsOHEX2/HtxpkGtXDjJKPHq43M2MR3Bp9+YUBAxcj5WMsGcs0lp7hFP6xADEJAcLdfta3SJCdNTa0Vw==</ds:SignatureValue>
        <ds:KeyInfo>
            <ds:X509Data>
                <ds:X509Certificate>
                   CertificateStuff...
                </ds:X509Certificate>
            </ds:X509Data>
        </ds:KeyInfo>
    </ds:Signature>
</facturaElectronicaCompraVenta>

I need to sign the xml file without the "ds" namespace like the following:

<facturaElectronicaCompraVenta xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="facturaElectronicaCompraVenta.xsd">
   <header></header>
   <detail></detail>
   <Signature xmlns="http://www.w3.org/2000/09/xmldsig#">
       <SignedInfo>
           <CanonicalizationMethod Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315"/>
           <SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256"/>
           <Reference URI="">
               <Transforms>
                   <Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/>
                   <Transform Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315#WithComments"/>
               </Transforms>
               <DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"/>
            <DigestValue>WmFvnKBZIr9D37PaYuxM3aoXVu9nDZT+2MI1I+RUh8s=  </DigestValue>
       </Reference>
       </SignedInfo>
       <SignatureValue> itb123fGGhh12DpFDFas34ASDAPpSSSSadasDasAS1smkRsj5ksdjasd8asdkasjd8asdkas8asdk21v a1qf+kBKLwF39mj+5zKo1qf+kBKLD42qD/+yxSMMS6DM5SywPxO1oyjnSZtObIe/45fdS4sE9+aNOn UncYUlSDAPpSSSSadasgIMWwlX2XMJ4SDAPpSSSSadas6qihJt/3dEIdta1RETSDAPpSSSSadas9S2W ALbT3VV8pjLqikVLcSDAPpSSSSadaseCKG8abcdssM0Wm8p+5grNNpSDAPpSSSSadasy4TvT4C3xS 70zSbKWeBUUglRcU8FECEcacu+UJaBCgRW0S3Q== </SignatureValue>
       <KeyInfo>
           <X509Data>
               <X509Certificate>
                  CertificateStuff..
               </X509Certificate>
           </X509Data>
       </KeyInfo>
   </Signature>
</facturaElectronicaCompraVenta>

I not sure why the site do not recognize the signature with the "ds:" namespace. I have previously struggled with xml namespaces and I do not understand them very well.

But, how could I sign the XML file without the "ds:" namespace without changing the signxml library source code?

Answer 1

I modify the codes from this answser from the following two aspects:

1. I use a recursive function. The original answer only reset nsmap for the root node. So there will be a xmlns:ds=***" left in the Signature Element(What you meet as you stated in the comments)
2. I add codes to update the attribte of the root. If using the original answer, the xsi:noNamespaceSchemaLocation attribute will be deleted.

def change_namsespace(root, target_nt):
    """
    Only keep the namespace target_nt in the tree rooted at the root
    To do so, I
    1. change the tag of root
    2. change the attribute
    3. change the nsmap
    4. keep other property
    """
    # change the tag of root
    tag = etree.QName(root.tag)
    root.tag = '{%s}%s' % (target_nt, tag.localname)

    # change the attribute of root
    # if there are attributes belong to other namespace, update to namespace of the
    # attributes to target_nt
    root_attrib_dict = dict(root.attrib)
    new_attrib_dict = {}
    for key, value in root_attrib_dict.items():
        key_QName = etree.QName(key)
        if key_QName.namespace is not None:
            new_key = '{%s}%s' % (target_nt, key_QName.localname)
        else:
            new_key = key
        new_attrib_dict[new_key] = value

    # set the new nsmap
    # only keep the target_nt, set to default namespace
    new_nsmap = {None: target_nt}

    # make the updated root
    new_root = etree.Element(root.tag, attrib=new_attrib_dict, nsmap=new_nsmap)

    # copy other properties
    new_root.text = root.text
    new_root.tail = root.tail

    # call recursively
    for old_root in root[:]:
        new_root.append(change_namsespace(old_root, target_nt))

    return new_root

Answer 2

If you use signxml the following way (slightly modified from your example - specifically lines 1 and 2 of the method get_signed_xml(...)):

from lxml import etree
from signxml import XMLSigner
import sys

def get_xml_tree_root():
        root = etree.Element('facturaElectronicaCompraVenta')
        xml_header = etree.SubElement(root, 'header')
        xml_detail = etree.SubElement(root, 'detail')
        return root

def get_signed_xml(root):
        signature = etree.SubElement(root,'{http://www.w3.org/2000/09/xmldsig#}Signature',Id='placeholder',nsmap={ None: 'http://www.w3.org/2000/09/xmldsig#' })
        signed_root = XMLSigner(c14n_algorithm='http://www.w3.org/2001/10/xml-exc-c14n#').sign(
            root,
            key=open('example.key').read(),
            cert=open('example.pem').read()
        )
        return signed_root

if __name__ == '__main__':
    root = get_xml_tree_root()
    signed_root = get_signed_xml(root)
    signed_xml_file = open('signed.xml','wb')
    signed_xml_file.write(etree.tostring(signed_root,encoding='UTF-8'))
    signed_xml_file.close()
    print(etree.tostring(signed_root,encoding='UTF-8',pretty_print=True).decode('UTF-8'))

you get the following output:

<facturaElectronicaCompraVenta>
  <header/>
  <detail/>
  <Signature xmlns="http://www.w3.org/2000/09/xmldsig#">
    <ds:SignedInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
      <ds:CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/>
      <ds:SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256"/>
      <ds:Reference URI="">
        <ds:Transforms>
          <ds:Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/>
          <ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/>
        </ds:Transforms>
        <ds:DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"/>
        <ds:DigestValue>hdNz66gEKYxWCR0+FfES7...</ds:DigestValue>
      </ds:Reference>
    </ds:SignedInfo>
    <SignatureValue>bSsNZJX4...</SignatureValue>
    <KeyInfo>
      <X509Data>
        <X509Certificate>MIIC/zCCAeegAwIBA...</X509Certificate>
      </X509Data>
    </KeyInfo>
  </Signature>
</facturaElectronicaCompraVenta>

which has the ds: prefix removed from everything but the SignedInfo element and it's children. Which to me actually looks like a bug in XMLSign().

Thus I am pretty sure, that signxml is not able to give you right now what you are looking for. Unfortunately I have no idea what you could do otherwise...

Edit: I just recognised, that the verification-site you linked to does some kind of caching so that it actually verified an old version of the file I created with earlier code than the above.

If you run the output of the code I posted above through that validation on that site, it actually recognises the xml document to be signed. It looks like the ds: prefix on the SignedInfo element and it's children doesn't matter.

The validation still fails, but it looks like it is only because I used a self-signed certificate. The verification result actually is:

Documento : signed.xml
 x test - Firma no válida
     v Documento auténtico
     x Cadena de confianza
     v Firmado en el periodo de vigencia
     v Firmado con certificado no revocado

So, maybe signxml can actually do what you want.

Answer 3

You can do something like this:

signer = XMLSigner()
ns = {None: signer.namespaces['ds']}
signer.namespaces = ns

ref: https://github.com/XML-Security/signxml/issues/171