If I have a link to a CSV on Yahoo Finance: http://ichart.finance.yahoo.com/table.csv?s=LOW&d=4&e=29&f=2014&g=d&a=8&b=22&c=1981&ignore=.csv
how would I write a web scraper to download multiple files based on a list of symbols: [LOW, SPY, AAPL]
from StringIO import StringIO
from urllib2 import urlopen
for symbol in symbols:
f = urlopen ('http://www.myurl.com'+symbol+'therestoftheurl')
p = f.read()
d = StringIO(p)
f.close
Do I need to write the contents of the url to file, or will it download automatically into a directory?
You can use a method like this to download files:
import urllib2
file_name = "myfile.xyz"
u = urllib2.urlopen(url)
f = open(file_name, 'wb')
block_sz = 4096
while True:
buffer = u.read(block_sz)
if not buffer:
break
f.write(buffer)
f.close()
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.