简体   繁体   中英

python - 'AttributeError: 'NoneType' object has no attribute 'text' when web scraping

I have to scraping data from ebay.com, my scraping item's is: 'title of car', 'make', 'model', 'transmission' and 'price', I have scraped all of this item's but when write code of 'transmission' show me this error: 'AttributeError: 'NoneType' object has no attribute 'text'' and not scrape 'transmission' item..

code:

import requests
from bs4 import BeautifulSoup
import re

url = 'https://www.ebay.com/b/Cars-Trucks/6001?_fsrp=0&_sacat=6001&LH_BIN=1&LH_ItemCondition=3000%7C1000%7C2500&rt=nc&_stpos=95125&Model%2520Year=2020%7C2019%7C2018%7C2017%7C2016%7C2015'
res = requests.get(url)
soup = BeautifulSoup(res.text, 'html.parser')

ebay_cars = soup.find_all('li', class_='s-item')
for car_info in ebay_cars:

    title_div = car_info.find('div', class_='s-item__wrapper clearfix')
    title_sub_div = title_div.find('div', class_='s-item__info clearfix')
    title_p = title_sub_div.find('span', class_='s-item__price')
    title_tag = title_sub_div.find('a', class_='s-item__link')
    title_maker = title_sub_div.find('span', class_='s-item__dynamic s-item__dynamicAttributes1')
    title_model = title_sub_div.find('span', class_='s-item__dynamic s-item__dynamicAttributes2')
    title_trans = title_sub_div.find('span', class_='s-item__dynamic s-item__dynamicAttributes3')

    name_of_car = title_tag.text
    price_of_car = title_p.text
    maker_of_car = title_maker.text
    model_of_car = title_model.text
    trans_of_car = title_trans.text

I think the find function will be returning a list of elements, and you can't use.text on a list. you can try adding first=True as an argument in the find function or use list indexing to access the element you need

edit: sorry, realised you said nonetype not list - you might need to try running the javascript on the page first if there are things missing. I recommend requests-html for small jobs needing javascript

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM