AttributeError: 'NoneType' object has no attribute 'text' - tuples

Question

it throws me an AttributeError when i add .text to the end of the variables in the loop. i remove them and it prints out all the tags along with the info. i'm not sure why it keeps throwing the AttributeError.

Any help is appreciated TT

i've tried:

biz_name = result.find('span', attrs={'itemprop':'name'}).text

and

biz_name = result.find('span', attrs={'itemprop':'name'}).text[1:-1]

Here is one cell of the results:

<span itemprop="name">Efrain Jimenez Jr. General Contractor Inc.</span>

and the script:

import requests
from bs4 import BeautifulSoup
import pandas as pd
import csv

r = requests.get('https://www.yellowpages.com/search?search_terms=remodeling&geo_location_terms=New+York%2C+NY')

soup = BeautifulSoup(r.text, 'html.parser')

results = soup.find_all('div', attrs={'class':'info'})

records = []

for result in results:
    biz_name = result.find('span', attrs={'itemprop':'name'})
    biz_phone = result.find('div', attrs={'itemprop':'telephone'})
    biz_address = result.find('span', attrs={'itemprop':'streetAddress'})
    biz_city = result.find('span', attrs={'itemprop':'addressLocality'})
    biz_zip = result.find('span', attrs={'itemprop':'postalCode'})
    records.append((biz_name, biz_phone, biz_address, biz_city, biz_zip))

df = pd.DataFrame(records, columns=['biz_name', 'biz_phone', 'biz_address', 'biz_city', 'biz_zip'])

df.to_csv('Yp_Remodel.csv', index=False, encoding='utf-8')

Answer 1

Maybe not the fanciest answer, but is seems that in some cases, some of the values are 'None', therefore, you will get an error if you try to access its text. Try this, it worked for me.

import requests
from bs4 import BeautifulSoup
import pandas as pd
import csv

r = requests.get('https://www.yellowpages.com/search?search_terms=remodeling&geo_location_terms=New+York%2C+NY')
soup = BeautifulSoup(r.text, 'html.parser')
results = soup.find_all('div', attrs={'class':'info'})

records = []

for result in results:
    biz_name = result.find('span', attrs={'itemprop':'name'}).text if result.find('span', attrs={'itemprop':'name'}) is not None else ''
    biz_phone = result.find('div', attrs={'itemprop':'telephone'}).text if result.find('span', attrs={'itemprop':'telephone'}) is not None else ''
    biz_address = result.find('span', attrs={'itemprop':'streetAddress'}).text if result.find('span', attrs={'itemprop':'streetAddress'}) is not None else ''
    biz_city = result.find('span', attrs={'itemprop':'addressLocality'}).text if result.find('span', attrs={'itemprop':'addressLocality'}) is not None else ''
    biz_zip = result.find('span', attrs={'itemprop':'postalCode'}).text if result.find('span', attrs={'itemprop':'postalCode'}) is not None else ''
    records.append((biz_name, biz_phone, biz_address, biz_city, biz_zip))


df = pd.DataFrame(records, columns=['biz_name', 'biz_phone', 'biz_address', 'biz_city', 'biz_zip'])

df.to_csv('Yp_Remodel.csv', index=False, encoding='utf-8')