使用lxml和python请求进行爬取。

Question

Okay, I am at it again and really trying to figure this stuff out with lxml and python. The last time I asked a question I was using xpath and had to figure out how to make a change in case that the direct xpath source itself would change. I have edited my code to try to go after the class instead. I keep running into a problem with it pulling the address up in memory and not the text that I want. Before anyone says there is a library for what I want to do, this is not about that but, rather, allowing me to understand this code. Here is what I have so far but when I print it out I get an error and I can add [0] behind the print[0].text but it still give me nothing. Any help would be cool.

from lxml import html
import requests
import time



while True:
page = requests.get('https://markets.businessinsider.com/index/realtime-chart/dow_jones')
content = html.fromstring(page.content)
#This will create a list of prices:
prices = content.find_class('price')


print(prices.text)
time.sleep(.5)

Answer 1

Probably a formatting issue from posting but your while loop is not indented.

Try my code below:

while True:
    page = requests.get('https://markets.businessinsider.com/index/realtime-chart/dow_jones')
    content = html.fromstring(page.content)
    prices = content.find_class('price')
    #You need to access the 'text_content' method
    text = [p.text_content() for p in prices]
    for t in text:
        if not t.startswith(r"\"):  # Prevents the multiple blank lines
           print(t)
    time.sleep(0.5)