beautifulsoup find_all() class shortcut doesn't work

Question

I am trying to find all p tags with the class column .

<p class="column">This is a column</p>
<p class="column">More columns heh</p>

I tried doing:

soup.find_all(class_='column')

which returned []

Then I tried:

soup.find_all(attrs={'class': 'column'})

and got the right results.

Shouldn't these two statements be identical? What's the difference?

Answer 1

(This is my first answer on Stack overflow so I'm kind of nervous!)

As others said those two are exactly identical. The only problem is that you are using an old version of Beautiful Soup. As it says HERE .

In older versions of Beautiful Soup, which don't have the class_ shortcut, you can use the attrs trick mentioned above. Create a dictionary whose value for “class” is the string (or regular expression, or whatever) you want to search for.

Hope it helped!

Answer 2

The statements are exactly identical, and I was not able to reproduce your problem:

>>> from bs4 import BeautifulSoup
>>> soup = BeautifulSoup('''<div>
... <p class="column">This is a column</p>
... <p class="column">More columns heh</p>
... </div>''')
>>> soup.find_all(class_='column')
[<p class="column">This is a column</p>, <p class="column">More columns heh</p>]
>>> import bs4
>>> bs4.__version__
'4.1.3'

Note that the class_ argument was introduced in version 4.1.2, so do make sure you use a recent version of BeautifulSoup. From the Searching by CSS class section :

As of Beautiful Soup 4.1.2, you can search by CSS class using the keyword argument class_ :

Answer 3

>>> from bs4 import BeautifulSoup as BS
>>> soup = BS('''<p class="column">This is a column</p>
<p class="column">More columns heh</p>''')
>>> list1 = soup.find_all(class_='column')
>>> list2 = soup.find_all(attrs={'class': 'column'})
>>> list1 == list2
True

There's no difference. I'm not sure why it isn't working for you. Maybe your BeautifulSoup module is outdated? I also was not able to reproduce your problem.

Answer 4

I have the save situation like your post, and I find out my CentOS python version is 2.6.6, and BeautifulSoup 4.1.0 they say in their help doc:

"The examples in this documentation should work the same way in Python 2.7 and Python 3.2."

So I upgrade my python 2.6 to 2.7, according to this link:

" http://toomuchdata.com/2014/02/16/how-to-install-python-on-centos/ "

After upgrade finished, the soup became beautiful.