Using global map variable with BaseX collection

Question

I'm using BaseX (v9.3.2) XQuery v3.1. I have a XQuery script that evaluates all XML files in a BaseX collection. The collection has about 100 files, each one with a size between 30MB to 1.5GB and with the same XSD schema. The script for better performance uses the BaseX Map Module for mapping keys to values and get values quickly by key. I have declared several module-global variables of type map(*) . The crux is that the key values are only unique in one XML file. In order to make the key unique over the whole collection i've simply added base-uri($value),'_' as an additional part of the key:). It works fine. The main drawback i see, is the need of tunneling the base-uri through all consecutive function calls FooN($baseuri, $arg2) FooN-1($baseuri, $arg2)... Foo1($baseuri, $arg2) even if the $baseuri value is set with the first call FooN($baseuri, $arg2) but used only in the last function of the execution chain. To make it more clear see the XQuery Code below. Here the $basuri is lastly only needed in helper:Foo1 but we need to pass it as an argument in helper:Foo2 too.

My question is if there is a better approach with respect to the arguments tunneling and with no other drawbacks (performance, memory).

XQuery Code

xquery version "3.1" encoding "utf-8";

declare namespace helper="helperns";

declare variable $helper:root := db:open("BMW");

declare variable $helper:mapAS_ApplicationRecordElementByTypeRef as map(*) := map:merge(for $value in $helper:root//*:APPLICATION-RECORD-ELEMENT[*:TYPE-TREF/@DEST='APPLICATION-PRIMITIVE-DATA-TYPE'] return map:entry(concat(base-uri($value),'_',$value/*:TYPE-TREF/tokenize(text(),'/')[last()]), $value));
declare variable $helper:mapAS_ImplementationDataTypeRefByApplicationDatatypeRef as map(*) := ...


...

declare function helper:Foo1($applPrimitiveDataType as element(),$basuri as xs:string)
as element() {
  let $applRecordElement := map:get($helper:mapAS_ApplicationRecordElementByTypeRef, concat($basuri,'_',$applPrimitiveDataType/*:SHORT-NAME/text()))
...
  let $resolved := if($implDataTypeElement) then (
...
    )
    else (
      <RESOLVED_DATATYPE>
        <SHORT-NAME resolveinfo='not resolved (DB ERROR)'>unknown</SHORT-NAME>
      </RESOLVED_DATATYPE>  
    )
  return ($resolved)
};

...

declare function helper:Foo2($basuri as xs:string)
as element()* {
  let $applicationPrimitiveDataTypes := ...
  let $res :=
    for $applicationPrimitiveDataType in $applicationPrimitiveDataTypes
...
      order by $shortname
      return (
        if (not($implDataTypeRef)) then (
          let $dataTypeResolved := helper:Foo1($applicationPrimitiveDataType,$basuri)
          return(
            <MAPPING appldtname="{$shortname}" size="{$dataTypeResolved/*:SHORT-NAME/@size}" basedatatype="{$dataTypeResolved/*:SHORT-NAME}" resolveinfo="{$dataTypeResolved/SHORT-NAME/@resolveinfo}"/>
          )
        )
      )
  return($res)
};


let $mappingsForAllDocs :=
  for $doc in helper:getAllDocs()/doc
...
    let $missingmappings := helper:Foo2($doc/@baseuri)
    return (
      <FILE name="{$doc/@filename}" missing="{count($missingmappings)}">
        <MISSING-MAPPING>{$missingmappings}</MISSING-MAPPING>
...
      </FILE>
    )

let $allDocs :=
  for $doc in helper:getAllDocs()/doc
    order by $doc/@filename
    return (
      <FILE name="{$doc/@filename}"/>
    )

return($mappingsForAllDocs)

Answer 1

Have you considered updating the XML files (or generating a new set) with unique IDs? Once there are unique IDs, XML databases like BaseX are capable of fast lookups using the fn:id() function. For updating XML files, see https://docs.basex.org/wiki/XQuery_Update .