执行我的课程搜寻器时遇到问题

Question

I'm completely newbie to python when it comes to scrape any web data using class. So, apology in advance for any serious mistake. I've written a script to parse the text using a tag from wikipedia web site. I tried to write the code accurately from my level best but for some reason when i execute the code it throws error. The code and the error I'm having are given below for your kind consideration.

The script:

import requests
from lxml.html import fromstring

class TextParser(object):

    def __init__(self):
        self.link = 'https://en.wikipedia.org/wiki/Main_Page'
        self.storage = None

    def fetch_url(self):
        self.storage = requests.get(self.link).text

    def get_text(self):
        root = fromstring(self.storage)
        for post in root.cssselect('a'):
            print(post.text)

item = TextParser()
item.get_text()

The error:

Traceback (most recent call last):
  File "C:\Users\mth\AppData\Local\Programs\Python\Python35-32\testmatch.py", line 38, in <module>
    item.get_text()
  File "C:\Users\mth\AppData\Local\Programs\Python\Python35-32\testmatch.py", line 33, in get_text
    root = fromstring(self.storage)
  File "C:\Users\mth\AppData\Local\Programs\Python\Python35-32\lib\site-packages\lxml\html\__init__.py", line 875, in fromstring
    is_full_html = _looks_like_full_html_unicode(html)
TypeError: expected string or bytes-like object

Answer 1

You're executing the following two lines

item = TextParser()
item.get_text()

When you initialize TextParser , self.storage is equal to None. When you execute the function get_text() it's still equal to None. So that's why you get that error.

However, if you change it to the following. self.storage should get populated with a string rather than being none.

item = TextParser()
item.fetch_url()
item.get_text()

If you want to call the function get_text without calling fetch_url you can do it this way.

def get_text(self):
    self.fetch_url()
    root = fromstring(self.storage)
    for post in root.cssselect('a'):
        print(post.text)