How to extract text from an HTML div tag file with BeautifulSoup?

Question

My python code is as below:

import requests 
from bs4 import BeautifulSoup
flipurl = "https://www.flipkart.com/search?q=realme+7&otracker=search&otracker1=search&marketplace=FLIPKART&as-show=off&as=off"
r = requests.get(flipurl)
htmlContent = r.content
soup = BeautifulSoup(htmlContent,'html.parser')
 
#i scrape flipkart product price

price= soup.find_all("div",class_="_30jeq3 _1_WHN1")
print(price.get_text())

#**I got this error how I get text:**

  "ResultSet object has no attribute '%s'. You're probably treating a list of elements like a single element. Did you call find_all() 
when you meant to call find()?" % key
AttributeError: ResultSet object has no attribute 'get_text'. You're probably treating a list of elements like a single element. Did you call find_all() when you meant to call find()?

As you can see from the above snippet I have tried to extract all the text but all I get an error and None. please solve this

Answer 1

According to the BeautifulSoupdocumentation , find_all returns a list of elements. As such, you're calling price.get_text() on a list, which is causing the error since that method is only possessed by individual element instances.

AttributeError: ResultSet object has no attribute 'get_text'. You're probably treating a list of elements like a single element. Did you call find_all() when you meant to call find()?

The error message hints that you want to be calling this method on a single element rather than a collection.

If we print out your price variable, we get the following:

<div class="_30jeq3 _1_WHN1">₹15,499</div>
<div class="_30jeq3 _1_WHN1">₹13,499</div>
<div class="_30jeq3 _1_WHN1">₹15,499</div>
...

Assuming you want a list of the text inside each div, simply perform list comprehension on your results:

price_elements = soup.find_all("div",class_="_30jeq3 _1_WHN1")
prices_text = [p.get_text() for p in price_elements]

This will give you the following list

['₹15,499', '₹13,499', '₹15,499', '₹13,499', '₹19,999', '₹29,999', '₹29,999', '₹7,499', '₹7,499', '₹9,999', '₹8,999', '₹7,999', '₹7,999', '₹9,999', '₹8,999', '₹16,999', '₹16,999', '₹14,999', '₹14,999', '₹11,999', '₹8,999', '₹8,999', '₹12,999', '₹11,999']