C# LINQ query creating inefficient SQL compared to original query

Question

I'm using EF 6, ASP.NET on Windows

I have this SQL statement which looks like this:

SELECT M.STMTDT, M.stmtno, ISNULL(SUM(M.PayAmount),0) as Fee, ISNULL(SUM(A.Amount),0) as Adjustment, ISNULL(SUM(M.PayAmount) + SUM(A.Amount),0) as Total FROM MainData M
    LEFT OUTER JOIN Adjustments A
    ON M.STMTDT = A.STMTDT AND M.stmtno = A.Stmtno
where M.CID= '334R' AND YEAR(M.stmtdt) > year(getdate())-4
GROUP BY M.STMTDT, M.stmtno

It's a pretty standard query which a group, join, where and sums. I would like to create the same query in LINQ but it is proving difficult to do so.

This is my LINQ query:

var fourYearsAgo = DateTime.Now.AddYears(-4).Year;

var dataWithoutGrouping = from m in MainData
                              where m.CID == "334r" && m.STMTDT.Value.Year > fourYearsAgo
                              join a in Adjustments
                                on new {m.STMTDT, m.Stmtno} equals new {a.STMTDT, a.Stmtno} into grp
                              from ja in grp.DefaultIfEmpty()
                              select new {
                                                Dt = m.STMTDT,
                                                No = m.Stmtno,
                                                Fee = m.PayAmount,
                                                Adjustment = ja.Amount
                                            };

    var data = (from b in dataWithoutGrouping
                group b by new {b.Dt, b.No }into grp
                select new {
                   StatmentFee = grp.Sum(x => x.Fee),
                   StatementAdjustments = grp.Sum(x => x.Adjustment),
                   StatementDate = grp.FirstOrDefault().Dt,
                   StatementNo = grp.FirstOrDefault().No
                   }).ToList();

Which produces this SQL:

-- Region Parameters
DECLARE @p0 VarChar(1000) = '334r'
DECLARE @p1 Int = 2014
-- EndRegion
SELECT [t3].[value] AS [StatmentFee], [t3].[value2] AS [StatementAdjustments], (
    SELECT [t6].[STMTDT]
    FROM (
        SELECT TOP (1) [t4].[STMTDT]
        FROM [MainData] AS [t4]
        LEFT OUTER JOIN [Adjustments] AS [t5] ON ([t4].[STMTDT] = [t5].[STMTDT]) AND ([t4].[stmtno] = [t5].[Stmtno])
        WHERE ((([t3].[STMTDT] IS NULL) AND ([t4].[STMTDT] IS NULL)) OR (([t3].[STMTDT] IS NOT NULL) AND ([t4].[STMTDT] IS NOT NULL) AND ((([t3].[STMTDT] IS NULL) AND ([t4].[STMTDT] IS NULL)) OR (([t3].[STMTDT] IS NOT NULL) AND ([t4].[STMTDT] IS NOT NULL) AND ([t3].[STMTDT] = [t4].[STMTDT]))))) AND ((([t3].[stmtno] IS NULL) AND ([t4].[stmtno] IS NULL)) OR (([t3].[stmtno] IS NOT NULL) AND ([t4].[stmtno] IS NOT NULL) AND ((([t3].[stmtno] IS NULL) AND ([t4].[stmtno] IS NULL)) OR (([t3].[stmtno] IS NOT NULL) AND ([t4].[stmtno] IS NOT NULL) AND ([t3].[stmtno] = [t4].[stmtno]))))) AND ([t4].[CID] = @p0) AND (DATEPART(Year, [t4].[STMTDT]) > @p1)
        ) AS [t6]
    ) AS [StatementDate], (
    SELECT [t9].[stmtno]
    FROM (
        SELECT TOP (1) [t7].[stmtno]
        FROM [MainData] AS [t7]
        LEFT OUTER JOIN [Adjustments] AS [t8] ON ([t7].[STMTDT] = [t8].[STMTDT]) AND ([t7].[stmtno] = [t8].[Stmtno])
        WHERE ((([t3].[STMTDT] IS NULL) AND ([t7].[STMTDT] IS NULL)) OR (([t3].[STMTDT] IS NOT NULL) AND ([t7].[STMTDT] IS NOT NULL) AND ((([t3].[STMTDT] IS NULL) AND ([t7].[STMTDT] IS NULL)) OR (([t3].[STMTDT] IS NOT NULL) AND ([t7].[STMTDT] IS NOT NULL) AND ([t3].[STMTDT] = [t7].[STMTDT]))))) AND ((([t3].[stmtno] IS NULL) AND ([t7].[stmtno] IS NULL)) OR (([t3].[stmtno] IS NOT NULL) AND ([t7].[stmtno] IS NOT NULL) AND ((([t3].[stmtno] IS NULL) AND ([t7].[stmtno] IS NULL)) OR (([t3].[stmtno] IS NOT NULL) AND ([t7].[stmtno] IS NOT NULL) AND ([t3].[stmtno] = [t7].[stmtno]))))) AND ([t7].[CID] = @p0) AND (DATEPART(Year, [t7].[STMTDT]) > @p1)
        ) AS [t9]
    ) AS [StatementNo]
FROM (
    SELECT SUM([t2].[PayAmount]) AS [value], SUM([t2].[value]) AS [value2], [t2].[STMTDT], [t2].[stmtno]
    FROM (
        SELECT [t0].[STMTDT], [t0].[stmtno], [t0].[PayAmount], [t1].[Amount] AS [value], [t0].[CID]
        FROM [MainData] AS [t0]
        LEFT OUTER JOIN [Adjustments] AS [t1] ON ([t0].[STMTDT] = [t1].[STMTDT]) AND ([t0].[stmtno] = [t1].[Stmtno])
        ) AS [t2]
    WHERE ([t2].[CID] = @p0) AND (DATEPART(Year, [t2].[STMTDT]) > @p1)
    GROUP BY [t2].[STMTDT], [t2].[stmtno]
    ) AS [t3]

As you can see this SQL is wildly inefficient compared to the original statement.

Please can someone help me convert my LINQ to produce the original SQL shown above.

Also, no I cannot use SQL for unexplained reasons, sorry!

Edit

Example data:

MainData:

StmtDate| |StmtNo |Fee

---

2016-01-29| 00:00:00.000 |3124360 |25.00

2016-02-12| 00:00:00.000 |3124391 |50.00

2016-01-29| 00:00:00.000 |3124360 |600.00

2016-02-12| 00:00:00.000 |3124391 |75.00

Adjustments:

StmtDate | StmtNo| Adj

2016-01-29 |00:00:00.000 |3124360 0.00

2016-02-12 |00:00:00.000 |3124391 0.00

2016-01-29 |00:00:00.000 |3124360 120.00

2016-02-12 |00:00:00.000 |3124391 60.00

Desired result:

StmtDate | StmtNo |Fee | Adj |Total

2016-01-29 00:00:00.000 |3124360 |25.00 |0.00 |0.00

2016-02-12 00:00:00.000 |3124391 |50.00 |0.00 |0.00

2016-02-19 00:00:00.000 |3124404 |600.00 |120.00 |720.00

2016-02-19 00:00:00.000 |3124405 |75.00 |60.00 |135.00

Answer 1

This is the Result i got:

If you have already your EF for MainData and Adjustment

I created a list:

    List<MainData> mdata = new List<MainData>();
    List<Adjustments> adj = new List<Adjustments>();
    List<Result> resFinal = new List<Result>();

and the LINQ

var gety = DateTime.Now.AddYears(-4);
            var res = from h in mdata
                      join j in adj
                      on h.StmtNo equals j.StmtNo
                      select new Result { StmtDate = h.StmtDate, StmtNo = h.StmtNo, Fee = h.Fee, Adj = j.Adj, Total = (h.Fee * j.Adj) };
            resFinal = res.Cast<Result>().Where(x=>x.StmtDate > gety).ToList();

The Final Class Holder as a Result

 public class Result
    {
        public DateTime StmtDate { get; set; }
        public int StmtNo { get; set; }
        public double Fee { get; set; }
        public double Adj { get; set; }
        public double Total { get; set; }
    }

Answer 2

Thanks to Ivan in the comments I came up with this solution:

var fourYearsAgo = DateTime.Now.AddYears(-4).Year;

var dataWithoutGrouping = from m in MainData
                              where m.CID == "334r" && m.STMTDT.Value.Year > fourYearsAgo
                              join a in Adjustments
                                on new {m.STMTDT, m.Stmtno} equals new {a.STMTDT, a.Stmtno} into grp
                              from ja in grp.DefaultIfEmpty()
                              select new {
                                                Dt = m.STMTDT,
                                                No = m.Stmtno,
                                                Fee = m.PayAmount,
                                                Adjustment = ja.Amount
                                            };

    var data = (from b in dataWithoutGrouping
                group b by new {b.Dt, b.No }into grp
                select new {
                   StatmentFee = grp.Sum(x => x.Fee),
                   StatementAdjustments = grp.Sum(x => x.Adjustment),
                   StatementDate = grp.Key.Dt,
                   StatementNo = grp.Key.No
                   }).ToList();

I just needed to use grp.Key as the FirstOrDefault() was creating a select for each record.

Also, here is the answer in VB(I translated my question into C# as VB isn't that popular here and I translated it back) if anyone needs it:

 Dim fourYearsAgo = DateTime.Now().AddYears(-4).Year

    Dim dataWithoutGrouping = From m In dbContext.MainDatas
                              Where m.CID = "334r" AndAlso m.STMTDT.Value.Year > fourYearsAgo
                              Group Join a In dbContext.Adjustments
                                On New With {m.STMTDT, m.stmtno} Equals New With {a.STMTDT, a.Stmtno} Into Group
                              From ja In Group.DefaultIfEmpty()
                              Select New With {
                                                .Dt = m.STMTDT,
                                                .No = m.stmtno,
                                                .Fee = m.PayAmount,
                                                .Adjustment = ja.Amount
                                            }

    Dim data = (From b In dataWithoutGrouping
                Group b By grpKeys = New With {b.Dt, b.No} Into Group
                Select New With {
                   .StatmentFee = Group.Sum(Function(x) x.Fee),
                   .StatementAdjustments = Group.Sum(Function(x) x.Adjustment),
                   .StatementDate = grpKeys.Dt,
                   .StatementNo = grpKeys.No
                   }).ToList()

It generates this SQL:

DECLARE @p0 VarChar(1000) = '334r'
DECLARE @p1 Int = 2014
-- EndRegion
SELECT SUM([t2].[PayAmount]) AS [StatmentFee], SUM([t2].[value]) AS [StatementAdjustments], [t2].[STMTDT] AS [StatementDate], [t2].[stmtno] AS [StatementNo]
FROM (
    SELECT [t0].[STMTDT], [t0].[stmtno], [t0].[PayAmount], [t1].[Amount] AS [value], [t0].[CID]
    FROM [MainData] AS [t0]
    LEFT OUTER JOIN [Adjustments] AS [t1] ON ([t0].[STMTDT] = [t1].[STMTDT]) AND ([t0].[stmtno] = [t1].[Stmtno])
    ) AS [t2]
WHERE ([t2].[CID] = @p0) AND (DATEPART(Year, [t2].[STMTDT]) > @p1)
GROUP BY [t2].[STMTDT], [t2].[stmtno]