如何从文件名以日期开头的目录中仅加载最新文件？

Question

I have files in one directory/folder named:

1. 2022-07-31_DATA_GVAX_ARPA_COMBINED.csv
2. 2022-08-31_DATA_GVAX_ARPA_COMBINED.csv
3. 2022-09-30_DATA_GVAX_ARPA_COMBINED.csv

The folder will be updated with each month's file in the same format as above eg.:

• 2022-10-31_DATA_GVAX_ARPA_COMBINED.csv
• 2022-11-30_DATA_GVAX_ARPA_COMBINED.csv

I want to only load the most recent month's.csv into a pandas dataframe, not all the files. How can I do this (maybe using glob)?

I have seen this used for prefixes using:

dir_files = r'/path/to/folder/*'

dico={}

for file in Path(dir_files).glob('DATA_GVAX_COMBINED_*.csv'):
    dico[file.stem.split('_')[-1]] = file

max_date = max(dico) 

Answer 1

Glob the directory with the pattern for known files of interest. Sort (natural) on the basename.

from glob import glob as GLOB
from os.path import join as JOIN, basename as BASENAME

def get_latest(directory):
    if all_files := list(GLOB(JOIN(directory, '*_DATA_GVAX_ARPA_COMBINED.csv'))):
        return sorted(all_files, key=BASENAME)[-1]

print(get_latest('/Users/Cobra'))

Answer 2

You could try something like this:

import pandas as pd
from pathlib import Path


dir_files = r'/path/to/folder/*'

dico = {}

for file in Path(dir_files).glob('*DATA_GVAX_ARPA_COMBINED*.csv'):
    date_value = pd.to_datetime(file.name.split('_')[0], errors="coerce")
    if pd.notna(date_value):
        dico[date_value] = file

max_date = max(dico.keys())
filepath = dico[max_date]
print(f'{max_date} -> {filepath}')
# Prints:
#
# 2022-10-31 00:00:00 -> 2022-10-31_DATA_GVAX_ARPA_COMBINED.csv