My script is not retrieving an item for a list. I would appreciate it, if someone could help me understand why?

Question

I am required to change the xkcd project in the AtBS book to download the comics from a different website. Here is my script.

#! python3
# getwebcomic.py - Downloads every single smbc comic.

import requests, os, bs4

os.chdir('C:\\Users\\Bob\\Desktop\\')
url = 'https://smbc-comics.com' # starting url
os.makedirs('smbc', exist_ok=True) # store comics in ./smbc
noAbuse=0

for noAbuse in range(0, 5):
#while not url.endswith('#'):
    # Download the page.
    print('Downloading page %s...' % url)
    res = requests.get(url)
    res.raise_for_status()

    soup = bs4.BeautifulSoup(res.text, "html.parser")

    # Find the URL of the comic image.
    comicElem = soup.select('#cc-comicbody')
    print('I am finding it')
    print(comicElem)
    if comicElem == []:
        print('Could not find comic image.')
    else:
        print(comicElem[0].get('src'))
        print('I dont know why the .get is returning NONE!')
        print('It is there???')
        print('...and now it crashes')
        comicUrl = 'https//smbc-comics.com' + comicElem[0].get('src')
        print(comicUrl)
        # Download the image.

        print('Downloading image %s...' % (comicUrl))
        res = requests.get(comicUrl)
        res.raise_for_status()

        # Save the image to ./smbc
        imageFile = open(os.path.join('smbc', os.path.basename(comicUrl)), 
'wb')
       for chunk in res.iter_content(100000):
            imageFile.write(chunk)
        imageFile.close()

    # Get the Prev button's url.
   prevLink = soup.select('a[rel="prev"]')[0]
   url = 'http://smbc-comic.com' + prevLink.get('href')

print('Done.')

OutPut

Downloading page https://smbc-comics.com...

I am finding it
[<div id="cc-comicbody"><img border="0" id="cc-comic" 
 src="/comics/1524150658-20170419 (1).png" 
 title="You can also just use an infinite quantity of compasses as on-off switches."/><br/></div>]
None

I dont know why the .get is returning NONE! It is there??? ...and now it crashes

Traceback (most recent call last):
File "C:\\Users\\Bob\\PythonScripts\\getwebcomic.py", line 31, in <module>
comicUrl = 'https//smbc-comics.com' + comicElem[0].get('src')
TypeError: must be str, not NoneType

I can't seem to figure out why the .get method is returning none when the 'src' attribute is there. Any hints would be appreciated. I put in some extra print() to help me see what is happening when the script runs.

Answer 1

comicElem[0] is a division ( <div> ). It does not have an src attribute, and that is why .get returns None . You should try comicElem[0].img.get("src") instead, which returns "/comics/1524150658-20170419 (1).png" .