C# Linq get descendants on a subquery

Question

I've been banging my head on the desktop for the past couple of hours trying to decipher this issue.

I'm trying to query an XML file with Linq, the xml has the following format:

<MRLGroups>
<MRLGroup>
    <MarketID>6084</MarketID>
    <MarketName>European Union</MarketName>
    <ActiveIngredientID>28307</ActiveIngredientID>
    <ActiveIngredientName>2,4-DB</ActiveIngredientName>
    <IndexCommodityID>59916</IndexCommodityID>
    <IndexCommodityName>Cucumber</IndexCommodityName>
    <ScientificName>Cucumis sativus</ScientificName>
    <MRLs>
        <MRL>
            <PublishedCommodityID>60625</PublishedCommodityID>
            <PublishedCommodityName>Cucumbers</PublishedCommodityName>
            <MRLTypeID>238</MRLTypeID>
            <MRLTypeName>General</MRLTypeName>
            <DeferredToMarketID>6084</DeferredToMarketID>
            <DeferredToMarketName>European Union</DeferredToMarketName>
            <UndefinedCommodityLinkInd>false</UndefinedCommodityLinkInd>
            <MRLValueInPPM>0.0100</MRLValueInPPM>
            <ResidueDefinition>2,4-DB</ResidueDefinition>
            <AdditionalRegulationNotes>Comments.</AdditionalRegulationNotes>
            <ExpiryDate xsi:nil="true" />
            <PrimaryInd>true</PrimaryInd>
            <ExemptInd>false</ExemptInd>
        </MRL>
        <MRL>
            <PublishedCommodityID>60626</PublishedCommodityID>
            <PublishedCommodityName>Gherkins</PublishedCommodityName>
            <MRLTypeID>238</MRLTypeID>
            <MRLTypeName>General</MRLTypeName>
            <DeferredToMarketID>6084</DeferredToMarketID>
            <DeferredToMarketName>European Union</DeferredToMarketName>
            <UndefinedCommodityLinkInd>false</UndefinedCommodityLinkInd>
            <MRLValueInPPM>0.0100</MRLValueInPPM>
            <ResidueDefinition>2,4-DB</ResidueDefinition>
            <AdditionalRegulationNotes>More Comments.</AdditionalRegulationNotes>
            <ExpiryDate xsi:nil="true" />
            <PrimaryInd>false</PrimaryInd>
            <ExemptInd>false</ExemptInd>
        </MRL>
    </MRLs>
</MRLGroup>

So far i've created classes for the "MRLGroup" section of the file

 var queryMarket = from market in doc.Descendants("MRLGroup")
                          select new xMarketID
                          {
                              MarketID = Convert.ToString(market.Element("MarketID").Value),
                              MarketName = Convert.ToString(market.Element("MarketName").Value)
                          };
        List<xMarketID> markets = queryMarket.Distinct().ToList();

        var queryIngredient = from ingredient in doc.Descendants("MRLGroup")
                              select new xActiveIngredients
                              {
                                  ActiveIngredientID = Convert.ToString(ingredient.Element("ActiveIngredientID").Value),
                                  ActiveIngredientName = Convert.ToString(ingredient.Element("ActiveIngredientName").Value)
                              };
        List<xActiveIngredients> ingredientes = queryIngredient.Distinct().ToList();

        var queryCommodities = from commodity in doc.Descendants("MRLGroup")
                               select new xCommodities {
                                   IndexCommodityID = Convert.ToString(commodity.Element("IndexCommodityID").Value),
                                   IndexCommodityName = Convert.ToString(commodity.Element("IndexCommodityName").Value),
                                   ScientificName = Convert.ToString(commodity.Element("ScientificName").Value)
                               };
        List<xCommodities> commodities = queryCommodities.Distinct().ToList();

After i got the "catalogues" I'm trying to query the document against the catalogues to achieve some sort of "groups", after all this, i'm going to send this data to the database, the issue here is that the xml files are around 600MB each and i get the everyday, so my approach is to create catalogues and just send the MRLs to the database joined to the "header" table that contains the Catalogues IDs, here's what i've done so far but failed miserably:

 //markets
        foreach (xMarketID market in markets) {
            //ingredients
            foreach (xActiveIngredients ingredient in ingredientes) {
                //commodities
                foreach (xCommodities commodity in commodities) {
                    var mrls = from m in doc.Descendants("MRLGroup")
                               where Convert.ToString(m.Element("MarketID").Value) == market.MarketID
                               && Convert.ToString(m.Element("ActiveIngredientID").Value) == ingredient.ActiveIngredientID
                               && Convert.ToString(m.Element("IndexCommodityID").Value) == commodity.IndexCommodityID
                               select new
                               {
                                   ms = new List<xMRLIndividial>(from a in m.Element("MRLs").Descendants()
                                                                 select new xMRLIndividial{
                                                                     publishedCommodityID = string.IsNullOrEmpty(a.Element("PublishedCommodityID").Value) ? "" : a.Element("PublishedCommodityID").Value,
                                                                     publishedCommodityName = a.Element("PublishedCommodityName").Value,
                                                                     mrlTypeId = a.Element("MRLTypeID").Value,
                                                                     mrlTypeName = a.Element("MRLTypeName").Value,
                                                                     deferredToMarketId = a.Element("DeferredToMarketID").Value,
                                                                     deferredToMarketName = a.Element("DeferredToMarketName").Value,
                                                                     undefinedCommodityLinkId = a.Element("UndefinedCommodityLinkInd").Value,
                                                                     mrlValueInPPM = a.Element("MRLValueInPPM").Value,
                                                                     residueDefinition = a.Element("ResidueDefinition").Value,
                                                                     additionalRegulationNotes = a.Element("AdditionalRegulationNotes").Value,
                                                                     expiryDate = a.Element("ExpiryDate").Value,
                                                                     primaryInd = a.Element("PrimaryInd").Value,
                                                                     exemptInd = a.Element("ExemptInd").Value
                                                                 })

                               };
                    foreach (var item in mrls)
                    {
                        Console.WriteLine(item.ToString());
                    }
                }
            }
        }

If you notice i'm trying to get just the MRLs descendants but i got this error:

All i can reach on the "a" variable is the very first node of MRLs->MRL not all of them, what is going on?

If you guys could lend me a hand would be super!

Thanks in advance.

Answer 1

With this line...

from a in m.Element("MRLs").Descendants()

...will iterate through all sub-elements, including children of children. Hence your error, since your <PublishedCommodityID> element does not have a child element.

Unless you want to specifically return all child elements of all levels, always use the Element and Elements axis instead of Descendant and Descendants :

from a in m.Element("MRLs").Elements()

That should solve your problem.

However, your query is also difficult to read with the nested foreach loops and the multiple tests for the IDs. You can simplify it with a combination of LINQ and XPath:

var mrls =
    from market in markets
    from ingredient in ingredientes
    from commodity in commodities
    let xpath = $"/MRLGroups/MRLGroup[{market.MarketId}]" +
        $"[ActiveIngredientID={ingredient.ActiveIngredientId}]" +
        $"[IndexCommodityID={commodity.IndexCommodityID}]/MRLs/MRL"
    select new {
        ms =
            (from a in doc.XPathSelectElements(xpath)
            select new xMRLIndividial {
                publishedCommodityID = string.IsNullOrEmpty(a.Element("PublishedCommodityID").Value) ? "" : a.Element("PublishedCommodityID").Value,
                publishedCommodityName = a.Element("PublishedCommodityName").Value,
                mrlTypeId = a.Element("MRLTypeID").Value,
                mrlTypeName = a.Element("MRLTypeName").Value,
                deferredToMarketId = a.Element("DeferredToMarketID").Value,
                deferredToMarketName = a.Element("DeferredToMarketName").Value,
                undefinedCommodityLinkId = a.Element("UndefinedCommodityLinkInd").Value,
                mrlValueInPPM = a.Element("MRLValueInPPM").Value,
                residueDefinition = a.Element("ResidueDefinition").Value,
                additionalRegulationNotes = a.Element("AdditionalRegulationNotes").Value,
                expiryDate = a.Element("ExpiryDate").Value,
                primaryInd = a.Element("PrimaryInd").Value,
                exemptInd = a.Element("ExemptInd").Value
            }).ToList()
    };