通过 dict.items() 的迭代顺序

Question

TLDR: If I build a dictionary at two seperate times from the same data processed in the same way, should the order of dictionary.items() be the same each time?

Hello,

I have dictionary linked_strain_acc which has about 2000 keys (strain names) and each key has another dictionary as a value ( data ).

linked_strain_acc = {'strain1' : {'gcf' : ['gcf1', 'gcf'..],
                                  'key2' : val2,
                                  .........},
                    'strain2' :  {.........},
                    ..........
                    'strain2000' :  {.........}}
          

I am iterating over a key ( 'gcf' ) in each data dictionary, which contains a list of gcf ids. I'm using the gcf ids to build a url for scraping, after testing that it's not already been scraped.

directory = r'C:\Users\u03132tk\.spyder-py3\scrape_dsmz\zip_files'
count = 0
start = time.time()
#allows you to stop and start
current_files = os.listdir(directory)
for strain,data in linked_strain_acc.items():
    for gcf in data['gcf']:
        count+=1
        filename = f'{strain}__{gcf}.zip'
        if filename not in current_files:
            download_url = f'https://antismash-db.secondarymetabolites.org/output/{gcf}/{gcf}.zip'
            response = requests.get(download_url)
            with open(fr'{directory}\{filename}', "wb") as infile:
                infile.write(response.content)
            print (f'downloaded {strain}, {gcf}')
        else:
            print (f'{strain}, {gcf} already scraped')
        if count%50 == 0:
            print (f'downloaded {count} jsons - script has been running for {round((time.time() - start)/60, 1)} minutes')

Question

I have already scraped about 1500 of the gcf urls and downloaded the files (out of the 2000ish total). When I ran it again this morning, instead of printing '{strain}, {gcf} already scraped' for the first 1500 print statements, its alternating between a couple of '{strain}, {gcf} already scraped' print messages and 'downloaded {strain}, {gcf}' print statements. This implies that the order of the linked_strain_acc dictionary has changed.

I made this dictionary from a CSV file which was processed in exactly the same way each time to make linked_strain_acc . Why would the order of the dict change, or am I missing something? I know that dict key/val order isn't ordered by eg alphabet or size, but I though it would be maintained when it is built from exactly the same data.

Thanks!

Answer 1

In older versions, python was using string pools to efficiently store longer strings by pooling shorter common segments. Every time you create a string, it may change the pool, and hence the order. The strings you dynamically create in

download_url = f'https://antismash-db.secondarymetabolites.org/output/{gcf}/{gcf}.zip'

may change the pool depending on your starting point. For reference: https://en.wikipedia.org/wiki/String_interning