AttributeError: 'NoneType' object has no attribute 'children'

Question

from urllib.request import urlopen
from bs4 import BeautifulSoup

html = urlopen("http://www.pythonscraping.com/pages/page3.html")
bsObj = BeautifulSoup(html,"html.parser")

for child in bsObj.find("table",{"id":"giftlist"}).children:
    print(child)

Could anyone tell me what wrong with my code is? :((( What should i do next?

Answer 1

You should put the code in try-except block

from urllib.request import urlopen
from bs4 import BeautifulSoup

html = urlopen("http://www.pythonscraping.com/pages/page3.html")
bsObj = BeautifulSoup(html,"html.parser")

try:
    for child in bsObj.find("table",{"id":"giftlist"}).children:
        print(child)
except AttributeError as e:
    print(e)
except:
    print("An error has occured")

***In your case I have visited the website the id is not "giftlist", it's "giftLift" you have done a typing mistake and that's why find function is returning none type object.

Answer 2

I'm not sure whether you have already solved this problem posted 3 years ago, but you've made a small mistake, I'm assuming.

The id of the tag is not giftlist... It is giftList

Is your code from the book "Web Scraping With Python" from the O'Reilly series? I found the exact same code from that book, including this webpage, pythonscraping.com/pages/page3.html , which is posted by the author for the purpose of giving the readers a place to practice. Btw on the book it is also giftList, so I think you might have copied the code wrong

try this one now

for child in bsObj.find("table",{"id":"giftList"}).children:
print(child)

Answer 3

One option is to put the offending loop construct into a try, then handle the exception that appears when the interator returns None:

from urllib.request import urlopen
from bs4 import BeautifulSoup

html = urlopen("http://www.pythonscraping.com/pages/page3.html")
bsObj = BeautifulSoup(html,"html.parser")

try:
    for child in bsObj.find("table",{"id":"giftlist"}).children:
        print(child)
except AttributeError:
    # do what you want to do when bsObj.find() returns None

Or you could check the resulting list for None before entering the loop:

from urllib.request import urlopen
from bs4 import BeautifulSoup

html = urlopen("http://www.pythonscraping.com/pages/page3.html")
bsObj = BeautifulSoup(html,"html.parser")

result = bsObj.find("table",{"id":"giftlist"})
if result:
    for child in result.children:
        print(child)
else:
    # do what you want to do when bsObj.find() returns None

Answer 4

It's typo issue. I also meet the same problem. In the web page, the id name should be "id="gift L ist", not giftlist. It should work after modifying the id name. Try it.