处理列表索引错误

Question

The script I have written to parse name, address and phone number from yellowpage is working perfectly as long as I'm excluding phone number from print statement. If I try to print the three of them, it throws error showing "list index out of range". I can't find out the remedy myself. Here is what I have tried so far.

import requests
from bs4 import BeautifulSoup

url=requests.get("https://www.yellowpages.com/search?search_terms=Coffee%20Shops&geo_location_terms=San%20Francisco%2C%20CA&page=1")
soup=BeautifulSoup(url.text,'lxml')
for item in soup.findAll(class_="info"):
    name=item.findAll(class_="business-name")[0].text
    address=item.findAll(class_="adr")[0].text
    # phone=item.findAll(class_="phones")[0].text
    # print(name,phone,address)
    print(name,address)

Link for sir adam: " https://www.dropbox.com/s/pt9yk6y5zu9r0ag/For%20sir%20adam.txt?dl=0 "

Answer 1

When selecting name , phone and address it's better to use find which returns only the first match instead of findAll which returns a list of all matches.

As for your question, the first item in soup.findAll(class_="info") has no 'phones' tag and so item.findAll returns an empty list, which gives you an error when you try to select the first item.

You can handle this case either with a try - except or if - else block. Examlple :

for item in soup.findAll(class_="info"):
    name=item.find(class_="business-name").text
    address=item.find(class_="adr").text
    phone=item.find(class_="phones").text if item.find(class_="phones") else None
    print(name,phone,address)

Or if you insist on using findAll :

phone=item.findAll(class_="phones")[0].text if item.findAll(class_="phones") else None

Using a function :

def if_exist(item, item_class):
    pro=item.find(class_=item_class)
    if pro:
        return pro.text
    return ""

Example :

phone=if_exist(item, "phones")

Answer 2

import requests
from bs4 import BeautifulSoup

url=requests.get("https://www.yellowpages.com/search?search_terms=Coffee%20Shops&geo_location_terms=San%20Francisco%2C%20CA&page=1")
soup=BeautifulSoup(url.text,'lxml')

if __name__ == '__main__':
    for item in soup.findAll(class_="info"):
        name=item.findAll(class_="business-name")[0].text
        address = item.findAll(class_="adr")[0].text if len(item.findAll(class_="adr")) else 'No Address' 
        phone = item.findAll(class_="phones")[0].text if len(item.findAll(class_="phones")) else 'No Phones'
        print(name,phone,address)