Filling Forms and Downloading Files - Requests and Python

Question

I'm trying to scrap these page . I used requests library to fill the forms. These part is working because when I change to holidays dates or not a dd/mm/yyyy format in the Dt_Ref form, I am warned.

The objective is to download the results of the filled form.

1 - Fill the Forms 2 - Download the file that is generated

Here's the code.

*Ps, I'm working under a corporate proxy, so I used HTTPProxyAuth to login with my credentials.

import requests
from requests.auth import HTTPProxyAuth


params = {'Dt_Ref': '20/08/2017', 'TpInstFinanceiro': 'Debentures'}

company_proxy = {"http":"proxy.xxxxx/yyyyyy.pac"}
credenciais = HTTPProxyAuth("qqqqq", "wwwww")

url = 'http://www.anbima.com.br/reune/reune_down.asp'


r = requests.post(url, data = params, proxies = itau_proxy , auth = credenciais)

Any idea about how to download the file?

Answer 1

You've missed some important parameters in the post data. Try with this params dictionary:

params = {
    'TpInstFinanceiro':'DEB', 'Tipo':'1', 
    'Dt_Ref':'02/08/2017', 'saida':'csv'
}

If will you don't want a csv file you can change the params['saida'] value.