使用lxml和请求进行HTML爬取

Question

I was doing like this tutorial, http://docs.python-guide.org/en/latest/scenarios/scrape/ , to scrap an html table and it doesn't work well.

My code:

import requests
from lxml import html

page = requests.get('http://www.dti.ufv.br/horario/horario.asp?ano=2015&semestre=1&depto=MAT')
tree = html.fromstring(page.text)

vaga = tree.xpath('/html/body/center/table/tbody/tr[2]/td/table[2]/tbody/tr[108]/td[9]')
print vaga

I think the problem is with XPath... I did it like the tutorial said using Google Chrome but it's not like in tutorial. Anyone can help me get the right XPath? Thanks guys!

Answer 1

In HTML content, there is no tbody tag.

In code we are considering tbody tag to find target tag.

vaga = tree.xpath('/html/body/center/table/tbody/tr[2]/td/table[2]/tbody/tr[108]/td[9]')

This will always return empty list because tbody tag is not present in HTML content.

HTml content:

 <table width="760" border="0" cellspacing="0" cellpadding="0">
    <tr>
      <td><img src="img/topo.jpg" width="760" height="101"></td>
    </tr>
    <tr>
      <td background="img/conteudo.jpg"><p align="right"><img src="img/setas_voltar.jpg" width="8" height="7"> <font size="1"><strong><a href="javascript:history.back();">voltar</a>&nbsp;</strong></font></p>
        <TABLE WIDTH=100% BORDER=0 CELLSPACING=1 CELLPADDING=1>
        <TR>
          <TD align=center> <br>
              <font color="Black" size=2><b> Hor&aacute;rio de Aulas 2015/1</b></font><br>          </TD>
        </TR>
      </TABLE>