How do I save a webpage to the disk for later scraping using splinter library?

Question

I have the following python code using splinter library for searching a specific term in a website:

from splinter import Browser
browser = Browser()

browser.visit("http://decs.bvs.br/cgi-bin/wxis1660.exe/decsserver/?IsisScript=../cgi-bin/decsserver/decsserver.xis&interface_language=p&previous_page=homepage&previous_task=NULL&task=start")
browser.choose('search_language','p')
browser.fill('search_exp','costas')
element = browser.find_by_name("consult_button")
element.click()

And it works, the firefox page opens the page with the results. However I have not found a way to save those results as a html file to disk in orde to scrape them for terms. How do you save the webpage to disk using splinter?

Thanks in advance

Answer 1

You can open a file for writing and write browser.html to it:

with open('output.html', 'w') as f:
    f.write(browser.html.encode('utf-8'))

---

Note that I don't see the legitimate reason why you need to save the html for post-scraping. splinter (as is selenium ) is powerful in terms of locating elements. The library is not only for programmatic web-browsing, it can navigate, search, extract etc. See Finding elements .