简体繁体中英

Fastest way to generate the absolute paths of all the files in a folder recursively in Python?

原文 2022-10-11 08:23:29 0 1 python/ python-3.x/ windows/ list/ glob

My requirement is to generate a list of the absolute path of all the files in a Windows folder recursively.

I have tried glob.glob and glob.iglob but both take around 1.5 hours to generate the list.

Now it wouldn't be a problem if it was a one-off situation. This program would run daily - and this is where the problem starts.

After the first time run, I would only need to get the list of the latest modified files (after the last run, using the timestamp of that run). This is so that even if the first run takes 1.5 hours, the daily run should take a few minutes at max. Now there is no way I can get the timestamp of each individual file without going through each of the files which means I would need to go through the entire folder regardless and check for the timestamp on each of those.

Can I optimize this in any way? Sure ideally, the data would be arranged by date but it's not the case.

1 answers

I'm not sure if there are pure Python approaches that will be faster. You basically want to have a service running that keeps track of changes in real-time. In principle you could put the whole folder under version control (Git), and check the pending changes once per day.

Getting the absolute paths of all files in a folder, without traversing the subfolders

What's the fastest way to recursively search for files in python?

Getting a list of all files (with absolute path) recursively and their uid with Python

Getting paths to all files in a directory (but not absolute path) in Python 3

How to recursively move all files inside subdirectories to main folder? [python]

Python script recursively rename all files in folder and subfolders

Generate all paths in an efficient way using networkx in python

Get absolute paths of all files in a directory

Fastest way to delete all files in large volume in python?

Python - Any way to turn relative paths into absolute paths?

暂无

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

Related Question Getting the absolute paths of all files in a folder, without traversing the subfolders What's the fastest way to recursively search for files in python? Getting a list of all files (with absolute path) recursively and their uid with Python Getting paths to all files in a directory (but not absolute path) in Python 3 How to recursively move all files inside subdirectories to main folder? [python] Python script recursively rename all files in folder and subfolders Generate all paths in an efficient way using networkx in python Get absolute paths of all files in a directory Fastest way to delete all files in large volume in python? Python - Any way to turn relative paths into absolute paths?

Related Tags

Fastest way to generate the absolute paths of all the files in a folder recursively in Python?

Question

1 answers

solution1 0 2022-10-11 08:29:33

solution1
0 2022-10-11 08:29:33