How to fix IndexError: list index out of range while scraping

Question

I'm getting this error again and again:

Traceback (most recent call last):
  File ".\bing.py", line 31, in <module>
    s.write(str(title[i].text) + ',' + link[i].text + ',' + description[i].text[30:70] + '\n')
IndexError: list index out of range

What is the meaning of this error and why i getting this error?

Here is my code:

for url_p in url_pattren:   
    time.sleep(3)   
    url1 = 'https://www.bing.com/search?q=site%3alocalbitcoins.com+%27%40gmail.com%27&qs=n&sp=-1&pq=site%3alocalbitcoins+%27%40gmail.com%27&sc=1-31&sk=&cvid=9547A785CF084BAE94D3F00168283D1D&first=' + str(url_p) + '&FORM=PERE3'
    driver.get(url1)
    time.sleep(r)
    title = driver.find_elements_by_tag_name('h2')
    link = driver.find_elements_by_css_selector("cite")
    description = driver.find_elements_by_css_selector("p")
    items = len(title)
    with open('btc_gmail.csv','a',encoding="utf-8") as s:
        for i in range(items):
            s.write(str(title[i].text) + ',' + link[i].text + ',' + description[i].text[30:70] + '\n')

Thanks!

Any Solution?

Answer 1

You said lengths are as following.

title: 12 Link:10 Description: 10

Link and Description has equal length but title has different. You are looping on title which has more elements than both link and Description.

range(12) will result in [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] so when it tries to access Link[10] it will raise

IndexError: list index out of range

As link has only 10 element and index start from 0 so accessing Link[10] will result in above exception.

Hope this help you.