os.listdir（）是确定性的吗？

Question

From Python's doc, os.listdir() returns

a list containing the names of the entries in the directory given by path. The list is in arbitrary order.

What I'm wondering is, is this arbitrary order always the same/deterministic? (from one machine to another, or through time, provided the content of the folder is the same)

Edit: I am not trying to make it deterministic, nor do I want to use this. I was just wondering (for example, what does the order depend on?)

Answer 1

In order to understand what is going on we can inspect the underlying implementation for python 3.2 that can be found here .

We will focus on the POSIX part that starts at line 2574 . In the code are defined:

DIR *dirp;              // will store the pointer to the directory
struct dirent *ep;      // will store the pointer to the entry

There are two important POSIX calls: opendir at line 2596 and readdir at line 2611 .

As you can read from the readdir man page:

The readdir() function returns a pointer to a dirent structure representing the next directory entry in the directory stream pointed to by dirp. It returns NULL on reaching the end of the directory stream or if an error occurred.

So, readdir reads the next entry in the directory, but it is up to the file system implementation to define what is the next . You can read more about this topic here :

[...] Because this is a per-filesystem thing, it follows that the traversal order can be different for different directories on the same system even if they have the same entries created in the same order, either because the directories are using different filesystem types or just because some parameters were set differently on the different filesystems.

Answer 2

You can look at the link posted in the comments by @Hamish, which digs a little into Python's hooks into UNIX' opendir and readdir implementations, from where you would need to dig deeper into file systems and how directory data structures are stored...

The short version of it however is trivially simple: the underlying file system does not store directory entries ordered by file names. It is concerned with keeping the directory entries sane and consistent, file names are just an arbitrary label associated with each entry and irrelevant to the core function of the file system. Worrying about the human readable labels associated with each directory entry is done at a higher level, for example in your Python code.

Yes, it is deterministic, it's certainly not purposefully randomised . However, the determinism is somewhere deep in the details of the file system implementation, and the lexical order of the file names plays no role in it.

Answer 3

It will probably depend on file system internals. On a typical unix machine, I would expect the order of items in the return value from os.listdir to be in the order of the details in the directory's "dirent" data structure (which, again, depends on the specifics of the file system).

I would not expect a directory to have the same ordering over time, if files are added and deleted.

I would not expect two "directories with the same contents" on two different machines to have a consistent ordering, unless specific care was taken when copying from one to the other.

Depending on a variety of specifics, the ordering may change on a single machine, over time, without any explicit changes to the directory, as various file system compacting operations take place (although I don't think I've seen a file system that would actually do this, but it's definitely something that could be done).

In short, if you want any sort of ordering you can reason about, sort the results, somehow. Then you have the guarantee that the ordering will be whatever your sorting imposes.