抓取数据python lxml

Question

I'm trying to retrieving a specific string by scraping. However it seem to return nothing. i'm using python and lxml, but not seem to return the string inside the a tag.

here is the html i'm trying to retrieve

<fieldset>
    <legend align="center">
        <a href="/counterstrike/events/302-cs-go-champions-league">CS:GO Champions League</a>
    </legend>
</fieldset>

Here is what i've tried

def get_league(self):
    request = requests.get(self.url)
    tree = html.fromstring(request.content)
    league = tree.xpath("//legend[@class='center']//a")
    return league

Answer 1

Use xpath to select the text explicitly

//legend[@align='center']/a/text()

This plugin for chrome helps a lot when writing lxml queries Xpath Helper

Answer 2

Try this, it's not lxml but you can use it for scraping purposes. Firstly I'm going to define my own-made function, it'll be easier to scrape then

def getBetweenHTML(strSource, strStart,strEnd):
    start = strSource.find(strStart) + len(strStart)
    end = strSource.find(strEnd,start)
    return strSource[start:end]

Afterwards, I'm going to do this:

def get_league(self):
    import urllib2
    url = urllib2.urlopen(self.url).read()
    getBetweenHTML(url, '<a href="/counterstrike/events/302-cs-go-champions-league">',"</a>")

This worked for me, it's just an alternative. If it's not what you're looking for, tell me and I'll re-write it for lxml.