Python: Using Request POST to get website data

Question

I've spend a lot of time trying to get retroactive prices from the website below in a python Dataframe:

http://www2.bmf.com.br/pages/portal/bmfbovespa/boletim2/Ajustes2.asp

I'm able to successfully get current data using:

url = 'http://www2.bmf.com.br/pages/portal/bmfbovespa/boletim2/Ajustes2.asp'
df = pd.read_html(url, header = 0)

now I'm strugguling with retroactive prices:

import pandas as pd
import requests

#BMF webpage
url = 'http://www2.bmf.com.br/pages/portal/bmfbovespa/boletim2/Ajustes2.asp'

#requests to manipulate BMF Form
r = requests.post(url, data = {'txtData':'11/21/2018'})
df_1 = pd.read_html(r.url, header = 0)

And I believe I have identified the correct id on the website mentioned:

.
.
.
<h1>Settlement Prices</h1>
<table border="0" cellpadding="0" cellspacing="0">
<form name="frmBD" onSubmit="return retroativo_Validator(this)"action="Ajustes2.asp" method="POST">
<tr>
<td class="TXT_Azul">RETROACTIVE:&nbsp;</td>
<td>
<input name="txtData" MAXLENGTH="10" onKeyPress="javascript:mask_data_home('2','0',true,frmBD);" type="text" class="combo" id="txtData">
<input name="Consultar" type="image" src="http://www.bmf.com.br/bmfbovespa/images/comum/btoOk.gif" border="0" align="absmiddle">
<img src="http://www.bmf.com.br/bmfbovespa/images/comum2/btoLegenda.gif" align="absmiddle" onMouseOver="javascript:legenda.style.display='block';" onMouseOut="javascript:legenda.style.display='none';">
<div id="legenda" style="position:absolute; width:200px; height:115px; z-index:10; background-color: #FFFFFF; layer-background-color: #FFFFFF; border: 1px solid #C9C9C9; overflow: visible; display: none;"> 
.
.
.

I really appreciate any help or insight

Thank you!

Answer 1

将以下标头添加到您的请求中：

Content-Type: application/x-www-form-urlencoded

Answer 2

Change df_1 = pd.read_html(r.url,header = 0) to df_1 = pd.read_html(r.text,header = 0) , because you desire the data after post request. If you access r.url that you will get the latest data becaue txtData is None.