how we can take value from website by using python code?

Question

import urllib
from datetime import date,timedelta
import datetime
import re
list =["infy.ns","grasim.ns","idea.ns","asianpain.ns","bajaj-auto-eq.ns",
       "drreddy.ns","boschltd.ns","kotakbank.ns","M&M.ns","ultracemc.ns",
       "sunpharma.ns","lt.ns","acc.ns","sbin.ns","bhartiartl.ns",
       "lupin.ns","reliance.ns","hdfcbank.ns","zeel.ns","ntpc.ns",
       "icicibank.ns","cipla.ns","tcs.ns","bpcl.ns","heromotoc.ns"]
i=0
while i<len(list):
    url="http://finance.yahoo.com/q?s="+list[i]+"&ql=1"
    htmlfile = urllib.urlopen(url)
    htmltext=htmlfile.read()
    regex='<span id="yfs_l84_'+list[i]+'">(.+?)</span>'
    pattern = re.compile(regex)
    price = re.findall(pattern,htmltext)
    print(price)
    i=i+1

i have to take value from finance.yahoo.com when i run that code by using terminal then i got all value on terminal but i want to put that value in my desktop text file

Answer 1

The easiest way requires no coding. Simply redirect the script's output onto a file, eg

python yahoo_scraper.py > prices.txt

or

python yahoo_scraper.py >> prices.txt

to append to an existing file.

Doing it in Python is also easy. Open a file for writing and write to it:

with open('prices.txt', 'w') as price_file:
    i=0
    while i<len(list):
        url="http://finance.yahoo.com/q?s="+list[i]+"&ql=1"
        htmlfile = urllib.urlopen(url)
        htmltext=htmlfile.read()
        regex='<span id="yfs_l84_'+list[i]+'">(.+?)</span>'
        pattern = re.compile(regex)
        price = re.findall(pattern,htmltext)
        print(price, file=price_file)
        i=i+1

Be aware that this will overwrite the file each time the script is run. If you want to append to the end of the file, open it in append mode by replacing 'w' with 'a' .

Your while loop would be better written as a for loop. Here is an example - I have assumed that list is renamed to stocks to avoid shadowing the builtin list :

stocks = ["infy.ns","grasim.ns",....]

with open('prices.txt', 'w') as price_file:
    for stock in stocks:
        url = "http://finance.yahoo.com/q?s={}&q1=1".format(stock)
        html = urllib.urlopen(url).read()
        pattern = r'<span id="yfs_l84_{}>(.+?)</span>'.format(stock)
        price = re.findall(pattern, html)
        print(price, file=price_file)

You might need to change the last line to print the first element of the list returned by re.findall() .