我想使用 Python 从网页中提取 CSV 文件。 网页抓取

Question

I want to take the.csv file, or the.xlsx file from this webpage. I thought about using webscraping, using beautifulsoup, but this seems inefficient. I want to be able to write a function that, when this webpage is called, the code locates the links to the CSV files and returns the CSV file to me.

This is so that I can then follow an analysis on the CSV file.

Please could someone help me out here!

Here's the link: https://data.london.gov.uk/dataset/recorded_crime_rates

Answer 1

Use the urllib library to get the source of a webpage, .

This seems to work:

import urllib.request, urllib.error, urllib.parse

url = 'https://data.london.gov.uk/dataset/recorded_crime_rates'
csvfile = r"C:\Tmp\CrimeRates.csv"

#open main page
response = urllib.request.urlopen(url)
webContent = response.read()
wc = str(webContent)

#get csv URL
i = wc.find(r"crime%20rates.csv")
i2 = wc.find("/download/recorded_crime_rates", i-200)
csvURL = "https://data.london.gov.uk" + wc[i2:i+17]
print(csvURL)

#get csv
csvresp = urllib.request.urlopen(csvURL)
csvdata = str(csvresp.read())
print(len(csvdata), "bytes")

#save csv to file
print("Saving To", csvfile)
f = open(csvfile,"w")
f.write(csvdata.replace(r"\r\n","\n"))
f.close()