How do I get rid of the TypeError

Question

I'm trying to build a web scraper and the weird thing is, sometimes the code works and sometimes it doesn't, without changing anything, it might be the websites problem but how could I fix it so it would work at all times?

I've tried rebuilding line 31 multiple times, yet it doesn't seem to work no matter how I do it.

# html parsing
page_soup = soup(page_html, "html.parser")

# grabs each appartment
containers = page_soup.findAll("div", {"class":"list-item-container"})

filename = "asunnot.csv"
f = open(filename, "w")

headers = "Kohdetta Vuokraa, Huoneistot, Talotyyppi ja Koko, Sijainti, Vapautuu, Vuokra"

f.write(headers)
count = 0
for page in range(1,10):
    my_url = "https://www.vuokraovi.com/vuokra-asunnot/Uusimaa?page={}&pageType="
    for container in containers:

        Vuokranantaja = container.findAll("div", {"class":"hidden-xs col-sm-3 col-4"})[0].img["alt"]

        Huoneistot = container.findAll("li", {"class":"semi-bold"})[1].text

        Talotyyppi = container.findAll("li", {"class":"semi-bold"})[0].text

        Sijainti = container.findAll("div", {"class":"hidden-xs col-sm-4 col-3"})[0].findAll("span", {"class":"address"})[0].text.strip().replace("\r", "").replace("\n", "").replace(" ", "").replace(",", ", ")

        Vapautuu = container.findAll("div", {"class":"hidden-xs col-sm-4 col-3"})[0].findAll("span", {"class":"showing-lease-container hidden-xs"})[0].li.text

        Vuokra = container.findAll("li", {"class":"rent"})[0].text.strip()

Desired output would be to give me the stuff I am trying to scrape, but instead it's giving me this:

Traceback (most recent call last):
  File "C:\Users\----\Desktop\vuokraovi.py", line 31, in <module>
    Vuokranantaja = container.findAll("div", {"class":"hidden-xs col-sm-3 col-4"})[0].img["alt"]
TypeError: 'NoneType' object is not subscriptable

Answer 1

Just sometimes there is nothing to scrape with findAll method. Check exception handling in documentation: https://docs.python.org/3/tutorial/errors.html#handling-exceptions

Answer 2

The error you get, 'NoneType' is not subscriptable , means you're trying to access something by index on None , ie something like None[idx] .

So, before trying to access item [0] on a container.findAll() result, you should first check whether there is something or not.