简单脚本运行非常慢

Question

I wrote this simple script to check whether or not a set of Bitcoin addresses have had transactions. However I think it's running very slowly because it's processing 2 per second more or less. The file has over 60k addresses so... this is gonna take forever!

Is that ok?

import urllib2

f = open('bc', 'r')
output = open('output', 'w')

n = 1
for bc in f:
    url = "https://blockchain.info/address/" + bc
    aux = urllib2.urlopen(url).read()    
    print n
    if int(aux[aux.find("<td id=\"n_transactions\">") + 24]) > 0:
        text = str(n) + ' -- ' + bc
        output.write(text)
    n = n + 1

output.close()
f.close()

Answer 1

You're opening a URL for every single instance in the file... it's going to take time to load the request, perform the find, and then write to file. You're loading a webpage 60,000 times... it's going to take a while.

Not necessarily the scripts fault, it's also your connection, the site's speed, etc. Lot of variables in your ability to process the data.

Answer 2

I don't see any problems with the script. Every page-load operation will have an impact on the output rendering time.

Perhaps why don't you think about some kind of library to perform this operation.

See if Python's PANDAS - pandas.pydata.org can help you. The biggest benefit that PANDAS will bring with itself is it can be used very efficiently to merge multiple files and perform operations on it collectively. This way you can reduce the page executions which would further make the script little more faster.