了解如何从HTML文件提取数据

Question

I am trying to access the "Yield Curve Data" available on this page . It has a radio button which upon clicking "Submit" results in a zip File, from which I am looking to get the data. I am looking to get the data from the "Retrieve all data" Option. My code is as follows, and from the statement print result.read() I realize that result is actually a HTML Document. My difficult is in understanding how to extract the data from result as I don't see any data in this. I am confused as to where to go from here.

import urllib, urllib2
import csv
from StringIO import StringIO
import pandas as pd
import os
from zipfile import ZipFile

my_url = 'http://www.bankofcanada.ca/rates/interest-rates/bond-yield-curves/'
data = urllib.urlencode({'lastchange': 'all'}) 
request = urllib2.Request(my_url, data)
result = urllib2.urlopen(request)

Thank You

Answer 1

Your going to need to generate a post request to the following endpoint:

http://www.bankofcanada.ca/stats/results/csv

With the following form data:

lookupPage: lookup_yield_curve.php
startRange: 1986-01-01
searchRange: all

This should give you the file.

You may also need to fake your useragent.