简体   繁体   中英

python: get data from changing span class using lxml xpath

I want to extract 'Return On Assets' from wsj websites. However, my code is not robust enough to work in different conditions. I able to extract data for ticker 'SCGM' using the code below but fail for'AASIA' as <span class="marketDelta deltaType-negative">

from lxml import html
import requests

StockData =['SCGM','AASIA']
page_wsj1 = requests.get('http://quotes.wsj.com/MY/'+StockData[x]+'/financials')
wsj1 = html.fromstring(page_wsj1.content)
wsj_fig = wsj1.xpath('//span[@class="marketDelta noChange"]/text()')
ROA = wsj_fig[25]

No issue for SCGM but for AASIA, it did not work as the span class is changed. For SCGM, the html tags as below. Full link here

<tr> <td> <span class="data_lbl">Return on Assets</span> <span class="data_data"> <span class="marketDelta noChange">18.26</span> </span> </td> </tr>

For AASIA, the html tags as below . Full link here

<tr> <td> <span class="data_lbl">Return on Assets</span> <span class="data_data"> <span class="marketDelta deltaType-negative">-1.36</span> </span> </td> </tr>

How to have a code that work for both conditions or point straight to 'Return on Assets'?

//td[normalize-space(span) = "Return on Assets"]/span[@class = "data_data"]/span

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM