线程之间的Python哈希不同

Question

I'm using Python 3.6 on Windows and have a parallelizable task that includes computing string hashes. This is basically a minimal version of my problem:

#!/usr/bin/env python3
from joblib import Parallel, delayed


def hash_some(foo):
    return hash(foo)


def main():
    hashes = Parallel(n_jobs=10)(delayed(hash_some)(s) for s in ['a', 'a', 'a'])

    print(hashes)


if __name__ == '__main__':
    main()

Now, for some reason this prints, eg, the following:

[3220780809080710068, -561460911962106608, -1551910331007446174]

Where they clearly should all be the same.

The hashes don't always differ, and especially for a lower n_job value they often turn out the same but this is not guaranteed.

I know hash() uses a random seed per program invocation but why does it apparently use a different seed per thread? Is there any way I can set a fixed (but random) seed for all my threads? (I know about PYTHONHASHSEED=0 but I'd prefer to find an in-code solution)

Answer 1

As you've already explained, randomization of hash can be controlled with PYTHONHASHSEED, for more information take a read on this . Now, if you want to control the behaviour by code and not with the python interpreter options or export that env. var a possible solution could be something like this:

#!/usr/bin/env python3
import random
import os
from joblib import Parallel, delayed

os.environ['PYTHONHASHSEED'] = '0'

def hash_some(foo):
    return hash(foo)

def main():
    hashes = Parallel(n_jobs=10)(delayed(hash_some)(s) for s in 'a' * 10000)

    print(set(hashes))

if __name__ == '__main__':
    main()

If you comment the os.environ line you'll see the final set length won't be 1 anymore