How to get grouped values from List within List with LINQ

Question

I'm looking to retrieve a list of the sum of property values in a list that is itself a property of another list, grouped by properties in the parent list, using LINQ.

To explain, I have a list of offers in a market with a trading date and hour of the day for a range of products, and a list of price and quantity bands within each offer. My classes are:

public class Offer
{
    public DateTime TradingDate { get; set; }
    public int HourOfDay { get; set; }
    public string ProductName { get; set; }
    public List<Band> OfferBands { get; set; }
}

public class Band
{
    public decimal Price { get; set; }
    public double Quantity { get; set; }
}

And what I'm looking to retrieve is the sum of Quantity for a certain Price for each TradingDate and HourOfDay , for every ProductName .

I haven't come up with a working solution, but as a start I'm trying something like (with a List<Offer> offers containing all offers, to retrieve quantities where the offer price < $10):

List<double> quantities = offers.SelectMany(o => o.Bands).Where(b => b.Price < 10).Select(b => b.Quantity)

But I don't know how to GroupBy the TradingDate and HourOfDay and retrieve the sum of Quantity . There can be multiple Offer s with multiple OfferBand s for different products, with various combinations of offer Price s, and I just want to get sum of Quantity for all products at a certain price grouped by date and time.

I could achieve this programmatically but I would like a LINQ solution. Thanks for your help.

Edit:

What I forgot to mention is that, where there are no Quantity s at the specified Price for a TradingDate and HourOfDay I would like to retrieve double.NaN (or 0 ).

With example data List<Offer> offers containing six Offer s:

TradingDate | HourOfDay | ProductName |       OfferBands
===================================================================
01/01/2017  |     1     | Chocolate   | Price = 2, Quantity = 6
            |           |             | Price = 5, Quantity = 10
-------------------------------------------------------------------
01/01/2017  |     2     | Chocolate   | Price = 3, Quantity = 6
            |           |             | Price = 5, Quantity = 20
-------------------------------------------------------------------
02/01/2017  |     1     | Chocolate   | Price = 3, Quantity = 7
            |           |             | Price = 6, Quantity = 9
-------------------------------------------------------------------
01/01/2017  |     1     | Cake        | Price = 5, Quantity = 11
            |           |             | Price = 8, Quantity = 3
-------------------------------------------------------------------
01/01/2017  |     2     | Cake        | Price = 2, Quantity = 1
            |           |             | Price = 8, Quantity = 4
-------------------------------------------------------------------
02/01/2017  |     1     | Cake        | Price = 3, Quantity = 9
            |           |             | Price = 5, Quantity = 13
-------------------------------------------------------------------

Selecting a sum of quantities for a given price, grouped by date and time, would give a List<double> output:

Where price >= 5

{ 24, 24, 22 }

Where price = 2

{ 6, 1, double.NaN }

Where price = 3

{ double.NaN, 6, 16 }

...where the output is the sum of quantities for all products at the specified prices for 01/01/2017 hour 1, 01/01/2017 hour 2, and 02/01/2017 hour 1.

Hopefully that is clear to follow.

Answer 1

I believe I've been able to manage the groupings you are after, though I haven't done the summation of the (quantity)*(whatever price matches some condition), as hopefully that is something that you can customize however you need to.

To get things grouped, I had to use several nested projections and do each grouping individually (it was actually quite fun to work this out, the big sticking point is that LINQ's IGrouping isn't as straightforward to use as you might expect, so each time I grouped I did a projection with a Select):

var projected = offers.GroupBy(x => x.ProductName)
                                  .Select(x => new
                                  {
                                      ProductName = x.Key,
                                      Dates = x.GroupBy(y => y.TradingDate).ToList()
                                                            .Select(y => new
                                                            {
                                                                TradingDate = y.Key,
                                                                Times = y.GroupBy(z => z.HourOfDay).ToList()
                                                                                      .Select(zx => new
                                                                                      {
                                                                                          Time = zx.Key,
                                                                                          Items = zx.ToList()
                                                                                      })
                                                            })
                                  }).ToList();

Hopefully, this will give you enough to start on for doing your summation with whatever extra checks you need for 0 items, prices not high enough, and so on.

Note that this query is probably not the most efficient if you're working directly with a database - it probably pulls more information than it really needs to at this point. I don't know enough about what you're working on to begin to optimize it, though.

Answer 2

        var offers = new List<Offer>();

        // flatten the nested list linq-style
        var flat = from x in offers
            from y in x.OfferBands
            select new {x.TradingDate, x.HourOfDay, x.ProductName, y.Price, y.Quantity};
        var grouped = from x in flat
            group x by new {x.TradingDate, x.HourOfDay, x.ProductName}
            into g
            select new
            {
                g.Key.TradingDate,
                g.Key.HourOfDay,
                g.Key.ProductName,
                OfferBands = (from y in g
                    group y by new {y.Price}
                    into q
                    select new {Price = q.Key, Quantity = q.Sum(_ => _.Quantity)}).ToList()
            };
        foreach (var item in grouped)
        {
            Console.WriteLine(
                    "TradingDate = {0}, HourOfDay = {1}, ProductName = {2}",
                    item.TradingDate,
                    item.HourOfDay,
                    item.ProductName);
            foreach (var offer in item.OfferBands)
                Console.WriteLine("    Price = {0}, Qty = {1}", offer.Price, offer.Quantity);
        }

Answer 3

First, you need to filter to get the desired Offer s with the matching OfferBands .

You can create/pass-in a filter if you want to make this a function, I will just define it inline:

Func<Band, bool> filter = (Band b) => b.Price == 3;

Since you don't care about ProductName , I used an anonymous type, but you could use Offer instead. At this point, we throw out the empty slots as well:

var filteredOffers = offers.Select(o => new { TradingDate = o.TradingDate, HourOfDay = o.HourOfDay, OfferBands = o.OfferBands.Where(filter).ToList() }).Where(gb => gb.OfferBands.Count > 0);

Now, since you want to include empty slots for TradingDate + HourOfDay that are in the original data but were filtered out, group the filtered data and create a dictionary:

var mapQuantity = filteredOffers.GroupBy(o => new { o.TradingDate, o.HourOfDay })
                                .Select(og => new { og.Key.TradingDate, og.Key.HourOfDay, QuantitySum = og.Sum(o => o.OfferBands.Sum(ob => ob.Quantity)) })
                                .ToDictionary(og => new { og.TradingDate, og.HourOfDay }, og => og.QuantitySum);

Then, going back to the original offers group find all the distinct slots ( TradingDate + HourOfDday ) and match them up to the QuantitySum , filling empty slots with double.NaN and convert to a List :

var ans = offers.Select(o => new { o.TradingDate, o.HourOfDay }).Distinct().OrderBy(g => g.TradingDate).ThenBy(g => g.HourOfDay).Select(g => mapQuantity.TryGetValue(g, out var sumq) ? sumq : double.NaN).ToList();

After re-thinking, I realized you could simplify by preserving the slots that are empty in the filteredOffers and then set their values after grouping:

var filteredOffers = offers.Select(o => new { TradingDate = o.TradingDate, HourOfDay = o.HourOfDay, OfferBands = o.OfferBands.Where(filter).ToList() });

var ans = filteredOffers.GroupBy(o => new { o.TradingDate, o.HourOfDay })
                        .OrderBy(og => og.Key.TradingDate).ThenBy(og => og.Key.HourOfDay)
                        .Select(og => (og.Sum(o => o.OfferBands.Count) > 0 ? og.Sum(o => o.OfferBands.Sum(ob => ob.Quantity)) : double.NaN));

By using the IGrouping Key to remember the slots, you can simplify the query:

var ans = offers.GroupBy(o => new { o.TradingDate, o.HourOfDay }, o => o.OfferBands)
                .OrderBy(obg => obg.Key.TradingDate).ThenBy(obg => obg.Key.HourOfDay)
                .Select(obg => {
                    var filteredOBs = obg.SelectMany(ob => ob).Where(filter).ToList();
                    return filteredOBs.Count > 0 ? filteredOBs.Sum(b => b.Quantity) : double.NaN;
                });

If you are willing to give up the double.NaN for zero instead, you can make this even simpler:

var ans = offers.GroupBy(o => new { o.TradingDate, o.HourOfDay }, o => o.OfferBands)
                .OrderBy(obg => obg.Key.TradingDate).ThenBy(obg => obg.Key.HourOfDay)
                .Select(obg => obg.SelectMany(ob => ob).Where(filter).Sum(b => b.Quantity));

Finally, to finish the dead horse off, some special extension methods can preserve the NaN returning property and use the simple query form:

public static class Ext {
    static double ValuePreservingAdd(double a, double b)  => double.IsNaN(a) ? b : double.IsNaN(b) ? a : a + b;
    public static double ValuePreservingSum(this IEnumerable<double> src) => src.Aggregate(double.NaN, (a, b) => ValuePreservingAdd(a, b));
    public static double ValuePreservingSum<T>(this IEnumerable<T> src, Func<T, double> select) => src.Select(s => select(s)).Aggregate(double.NaN, (a, b) => ValuePreservingAdd(a, b));
}

var ans = offers.GroupBy(o => new { o.TradingDate, o.HourOfDay }, o => o.OfferBands)
                .OrderBy(obg => obg.Key.TradingDate).ThenBy(obg => obg.Key.HourOfDay)
                .Select(obg => obg.SelectMany(ob => ob).Where(filter).ValuePreservingSum(b => b.Quantity));