Sort 40k images in a folder python

Question

import os
import time
from PIL import Image as PImage
import pathlib 
import glob

try:
    path = r"\\x.x.x.x\PVCTData\ELImages\2021_03_08"
    os.chdir(path)
    combo = os.listdir(path)
    combo.sort(key=os.path.getctime,reverse=True)
    print("done")
    print(combo)
    x=0
    loadedimages=[]
 
    for image in combo:
            print(image)
            img = PImage.open(os.path.join(path,image))
            loadedimages.append(img)
            x+=1
            if x>3:
                break
    print(loadedimages)
    loadedimages[0].show()
    loadedimages[1].show()
    loadedimages[2].show()
except Exception as ex:
    print(ex)

Assuming i have a shared folder with 40k pictures. I would like to sort the pictures based on the creation date as my machine will send new picture to the folder every 5 seconds. The code above can work but it is too slow to handle the amount of pictures which it took about 15 mins in order to sort and show. I only need to show the latest 60 pictures every 1 hour.

Answer 1

You have to filter the file list before sorting, such that any file you don't care about is not sorted. I recommend you move the files you don't need to another location (organized properly is a plus).

To do so use something like this before sorting:

combo = os.listdir()
currentTime = time.time()
timerange = 310
oldestOK = currentTime - timerange
filtered = list()
for file in os.listdir():
    if path.getctime(file) > oldestOK:
        filtered.append(file)

filtered.sort(key=os.path.getctime, reverse=True)

Answer 2

If you only need to use the last 60 items, why not just only loop over that part of the sort?
So instead of

for image in combo:

for image in combo[:60]:

Of course I'm not sure which part takes longer (the sorting, or handling the images after they are sorted).
And ofcourse what Christian stated, try to order them in more subdirectories after you've managed them.