Python sort list of lists partially reverse based on date

Question

I have a problem. I have list of lists that look something like this:

[
[datetime.date(2019, 3, 29), Decimal('44819.75')],
[datetime.date(2019, 3, 29), Decimal('45000.00')],
[datetime.date(2019, 3, 28), Decimal('0.00')],
[datetime.date(2019, 3, 22), Decimal('-275.00')],
[datetime.date(2019, 3, 22), Decimal('-350.00')],
[datetime.date(2019, 3, 22), Decimal('-175.00')]
]

I need sorting to be on date field(1st one), but each set of same dates must be sorted in reverse order. Resulting list must look like this:

[
[datetime.date(2019, 3, 29), Decimal('45000.00')],
[datetime.date(2019, 3, 29), Decimal('44819.75')],
[datetime.date(2019, 3, 28), Decimal('0.00')],
[datetime.date(2019, 3, 22), Decimal('-175.00')],
[datetime.date(2019, 3, 22), Decimal('-350.00')],
[datetime.date(2019, 3, 22), Decimal('-275.00')],
]

As you can see list is ordered by date but, for the same dates list is reversed.

dates are still descending 2019-3-29 2019-3-28 2019-3-22 but for each date, if there more than 1 element for that date, items are reversed.

for 2019-3-29 there are 2 element

[datetime.date(2019, 3, 29), Decimal('44819.75')],
[datetime.date(2019, 3, 29), Decimal('45000.00')],

and in resulting list of lists order is reversed

[datetime.date(2019, 3, 29), Decimal('45000.00')],
[datetime.date(2019, 3, 29), Decimal('44819.75')],

Unfortunately i can not find most pythonic way to do that, only ugly nested cycles

Answer 1

I took the liberty to simplify the datatypes since it is easier to read this way.

# Simplified representation.
# a few random values at the start and then multiple 2's and that the current order is a,b,c
# We expect all values to be sorted on the integer part first. And that the order for the 2's is c,b,a at the end.
data = [
    [1, '-'],
    [5, '-'],
    [3, '-'],

    [2, 'a'],
    [2, 'b'],
    [2, 'c']
]


data = data[::-1]
data = sorted(data, key=lambda x:x[0])

Printing the data will yield:

[1, '-']
[2, 'c']
[2, 'b']
[2, 'a']
[3, '-']
[5, '-']

Which I believe is that you wanted.

This solution is very easy to read which has its benefits when working with others.

sorted in python is a stable sorting algorithm. This is why you if you sort normally the order of 'abc' is preserved. Thats why reversing first works, sorted will not change the order in which equal items appeared.

Note that this also works.

data = sorted(data, key=lambda x:x[0], reverse=True)
data = data[::-1]

Here we do a reverse sort and then read the data backwards.

Answer 2

An O(n) solution using itertools.groupby to group and reverse each date's items:

data = [d for _, g in groupby(data, lambda d: d[0]) for d in [*g][::-1]]

(This requires the dates to already be descending in the input, but your question, especially your "dates are still descending" sounds like that's indeed the case.)

Demo:

import datetime
from decimal import Decimal
from itertools import groupby

data = [
[datetime.date(2019, 3, 29), Decimal('44819.75')],
[datetime.date(2019, 3, 29), Decimal('45000.00')],
[datetime.date(2019, 3, 28), Decimal('0.00')],
[datetime.date(2019, 3, 22), Decimal('-275.00')],
[datetime.date(2019, 3, 22), Decimal('-350.00')],
[datetime.date(2019, 3, 22), Decimal('-175.00')]
]

data = [d for _, g in groupby(data, lambda d: d[0]) for d in [*g][::-1]]

for d in data:
    print(d)

Output:

[datetime.date(2019, 3, 29), Decimal('45000.00')]
[datetime.date(2019, 3, 29), Decimal('44819.75')]
[datetime.date(2019, 3, 28), Decimal('0.00')]
[datetime.date(2019, 3, 22), Decimal('-175.00')]
[datetime.date(2019, 3, 22), Decimal('-350.00')]
[datetime.date(2019, 3, 22), Decimal('-275.00')]

Answer 3

You could sort by date, which reverses the dates but for each date keeps the order of the items (because it's a stable sort). And then reverse the whole thing, so your dates are descending again but each date's items are reversed.

data.sort(key=lambda d: d[0])
data.reverse()

Demo:

import datetime
from decimal import Decimal

data = [
[datetime.date(2019, 3, 29), Decimal('44819.75')],
[datetime.date(2019, 3, 29), Decimal('45000.00')],
[datetime.date(2019, 3, 28), Decimal('0.00')],
[datetime.date(2019, 3, 22), Decimal('-275.00')],
[datetime.date(2019, 3, 22), Decimal('-350.00')],
[datetime.date(2019, 3, 22), Decimal('-175.00')]
]

data.sort(key=lambda d: d[0])
data.reverse()

for d in data:
    print(d)

Output:

[datetime.date(2019, 3, 29), Decimal('45000.00')]
[datetime.date(2019, 3, 29), Decimal('44819.75')]
[datetime.date(2019, 3, 28), Decimal('0.00')]
[datetime.date(2019, 3, 22), Decimal('-175.00')]
[datetime.date(2019, 3, 22), Decimal('-350.00')]
[datetime.date(2019, 3, 22), Decimal('-275.00')]

Answer 4

In [4]: dates = [
   ...: [datetime.date(2019, 3, 29), Decimal('44819.75')],
   ...: [datetime.date(2019, 3, 29), Decimal('45000.00')],
   ...: [datetime.date(2019, 3, 28), Decimal('0.00')],
   ...: [datetime.date(2019, 3, 22), Decimal('-275.00')],
   ...: [datetime.date(2019, 3, 22), Decimal('-350.00')],
   ...: [datetime.date(2019, 3, 22), Decimal('-175.00')]
   ...: ]

In [5]: sorted(dates, key=lambda x: (x[0].day, x[1]), reverse=True)

Out[5]:
[[datetime.date(2019, 3, 29), Decimal('45000.00')],
 [datetime.date(2019, 3, 29), Decimal('44819.75')],
 [datetime.date(2019, 3, 28), Decimal('0.00')],
 [datetime.date(2019, 3, 22), Decimal('-175.00')],
 [datetime.date(2019, 3, 22), Decimal('-275.00')],
 [datetime.date(2019, 3, 22), Decimal('-350.00')]]

Answer 5

In Python, when you sort a list of iterables (like list, tuple, etc), element lists are sorted according to their values at first index. If they are same, then the values at the following index are compared.
The reversing of decimals are due to this.

The below is what you want:

... import datetime
... from decimal import Decimal
... from operator import itemgetter
... from itertools import groupby, chain
... 
... data = [
...     [datetime.date(2019, 3, 29), Decimal('44819.75')],
...     [datetime.date(2019, 3, 29), Decimal('45000.00')],
...     [datetime.date(2019, 3, 28), Decimal('0.00')],
...     [datetime.date(2019, 3, 22), Decimal('-275.00')],
...     [datetime.date(2019, 3, 22), Decimal('-350.00')],
...     [datetime.date(2019, 3, 22), Decimal('-175.00')]
... ]
... date_sorted_data = sorted(data, key=itemgetter(0), reverse=True)
... 
... result = list(
...     chain.from_iterable(
...         [
...             reversed(list(g)) 
...             for k, g in groupby(
...                 date_sorted_data, key=itemgetter(0)
...             )
...         ]
...     )
... )
... 
... print(result)
... 
[[datetime.date(2019, 3, 29), Decimal('45000.00')], [datetime.date(2019, 3, 29), Decimal('44819.75')], [datetime.date(2019, 3, 28), Decimal('0.00')], [datetime.date(2019, 3, 22), Decimal('-175.00')], [datetime.date(2019, 3, 22), Decimal('-350.00')], [datetime.date(2019, 3, 22), Decimal('-275.00')]]

Answer 6

Here's a way that uses an intermediate data structure:

>>> from collections import defaultdict

>>> dd = defaultdict(list)
>>> for x,y in data:
    dd[x].insert(0,y) #key is date, value is reverse list of Decimals for each date
>>> dd
defaultdict(<type 'list'>, {datetime.date(2019, 3, 29): [Decimal('45000.00'), Decimal('44819.75')], datetime.date(2019, 3, 28): [Decimal('0.00')], datetime.date(2019, 3, 22): [Decimal('-175.00'), Decimal('-350.00'), Decimal('-275.00')]})
>>> dataout = [[x,y] for x in sorted(dd.keys(),reverse=True) for y in dd[x]]
>>> dataout
[
 [datetime.date(2019, 3, 29), Decimal('45000.00')],
 [datetime.date(2019, 3, 29), Decimal('44819.75')],
 [datetime.date(2019, 3, 28), Decimal('0.00')],
 [datetime.date(2019, 3, 22), Decimal('-175.00')],
 [datetime.date(2019, 3, 22), Decimal('-350.00')],
 [datetime.date(2019, 3, 22), Decimal('-275.00')]
]

Answer 7

So you want to sort by date, and in case of tie sort by the reverse of the current position. Easy:

[value for _, value in sorted(enumerate(dates), key=lambda x: (x[1], -x[0]), reverse=True)]

Results:

>>> pprint([val for _, val in sorted(enumerate(seq), key=lambda x: (x[1][0], -x[0]), reverse=True)])
[[datetime.date(2019, 3, 29), Decimal('44819.75')],
 [datetime.date(2019, 3, 29), Decimal('45000.00')],
 [datetime.date(2019, 3, 28), Decimal('0.00')],
 [datetime.date(2019, 3, 22), Decimal('-275.00')],
 [datetime.date(2019, 3, 22), Decimal('-350.00')],
 [datetime.date(2019, 3, 22), Decimal('-175.00')]]