MySQL get the sum of all rows without retrieving all of them

Question

This may be a little confusing but please bear with me. Here's the thing:

I have a database that contains ~1000 records, as the following table illustrates:

+------+----------+----------+
| id   | date     | amount   |
+------+----------+----------+
| 0001 | 14/01/15 |      100 |
+------+----------+----------+
| 0002 | 14/02/04 |      358 |
+------+----------+----------+
| 0003 | 14/05/08 |     1125 |
+------+----------+----------+

What I want to do is this:

1. Retrieve all the records beginning at 2014 and until yesterday:

    WHERE `date` > '14-01-01' AND `date` < CURDATE() 

2. But also get the sum of amount up to the current date, this is:

    WHERE `date` < CURDATE() 

I've already got this working by just selecting all the records based on the second condition, getting the sum, and then excluding those which don't match the first condition. Something like this:

SELECT `id`, `date`, `amount` FROM `table`
WHERE `date` < CURDATE()

And then:

$rows = fetchAll($PDOStatement);

foreach($rows as $row) {
    $sum += $row->amount;
    if (
        strtotime($row->date) > strtotime('14-01-01') &&
        strtotime($row->date) < strtotime(date('Y-m-d'))
    ) {
        $valid_rows[] = $row;
    }
}
unset $rows;

Is there a way to achieve this in a single query, efficiently? Would a transaction be more efficient than sorting out the records in PHP? This has to be SQL-standard compliant (I'll be doing this on MySQL and SQLite).

Update:

It doesn't matter if the result ends up being something like this:

+------+----------+----------+-----+
| id   | date     | amount   | sum |
+------+----------+----------+-----+
| 0001 | 14/01/15 |      100 | 458 |
+------+----------+----------+-----+
| 0002 | 14/02/04 |      358 | 458 |
+------+----------+----------+-----+
| 0003 | 14/05/08 |     1125 | 458 |
+------+----------+----------+-----+

The worst case would be when the resulting set ends up being the same as the set that gives the sum (in this case appending the sum would be irrelevant and would cause an overhead), but for any other regular cases the bandwith save would be huge.

Answer 1

EDIT :

You can get the SUM using a LEFT OUTER JOIN.

    SELECT t1.`id`, t1.`date`, t2.sum_amount
    FROM
    `table` t1
    LEFT OUTER JOIN
    (
        SELECT SUM(`amount`) sum_amount
        FROM `table`
        WHERE `date` < CURDATE()
    ) t2
   ON 1 = 1
   WHERE t1.`date` > STR_TO_DATE('01,1,2014','%d,%m,%Y') AND t1.`date` < CURDATE();

Answer 2

You can create a special record with your sum and add it at the end of your first query

SELECT * FROM `table` WHERE `date` > '14-01-01' AND `date` < CURDATE()
UNION
SELECT 9999, CURDATE(), SUM(`amount`) FROM `table` WHERE `date` < CURDATE()

Then you will have all your desired record and the record with id 9999 or whatever is your sum

Answer 3

This could be achieved by correlated subquery, something like below:

SELECT *, (SELECT SUM(amount) FROM t WHERE t.date < t1.date) AS PrevAmount
FROM        t AS t1
WHERE `date` > '14-01-01' AND `date` < CURDATE()

However it is very unefficient if the number of records is large.

Answer 4

It's hackish, but:

> select * from foo;
+------+------+
| id   | val  |
+------+------+
|    1 |    1 |
|    2 |    2 |
|    3 |    3 |
|    4 |    4 |
|    5 |    5 |
+------+------+
5 rows in set (0.02 sec)

> select * from foo
left join (
    select sum(val)
    from foo
    where id < 3
) AS bar ON 1=1
where id < 4;
+------+------+----------+
| id   | val  | sum(val) |
+------+------+----------+
|    1 |    1 |        3 |
|    2 |    2 |        3 |
|    3 |    3 |        3 |
+------+------+----------+

Basically, do your summing in a joined subquery. That'll attach the sum result to every row in the outer table's results. You'll waste a bit of bandwidth sending that duplicated value out with every row, but it does get you the results in a "single" query.

Answer 5

This will do what you want it to do...optimizing the subquery is the real challenge:

SELECT id,date,amount,(SELECT SUM(amount) FROM table) AS total_amount
FROM table
WHERE date BETWEEN '14-01-01' AND DATE_ADD(CURDATE(), INTERVAL -1 DAY)