How can I get text from only <p> and <h2> tags when finding element by class with selenium and python?

Question

I am trying to get the text only from the h2 and the first p tag. I've been using class name to find the div and the output gives me all of the text in the div (obviously).

Here is the HTML:

<div class="horoscope-content">
<h2> Today's Libra Horoscope for January 27, 2022 <span class="today-badge">TODAY</span></h2>
<p>Go with the flow, Libra. If you find that a situation isn't unfolding the way you'd like it to, take it as a sign to back off. Swimming upstream is hard work, so use your energy more efficiently by exploring different options. When you step back from a stressful situation, circumstances could turn around. Lighten up by considering other possibilities or talking it through with a helpful friend.</p>            
<p>What's in the stars for you tomorrow? <a href="/horoscopes/daily/libra/friday">Read it now</a>.</p>
<div class="dropdown-inline">Read the <b>daily horoscope</b> for another zodiac sign:<div id="dropdown_below_horoscope_dropdown" class="dropdown">

Here is the code I'm using:

libra_content = driver.find_elements(By.CLASS_NAME, 'horoscope-content')

I assume the answer is to use xpath but I can't figure out how to include both tags. Do I need to use two separate lines of code to do it or can I combine both into one?

Answer 1

You could use:

For h2:

libra_content = driver.find_element_by_css_selector("div[class='horoscope-content'] > h2 ")

For p:

libra_content = driver.find_element_by_css_selector("div[class='horoscope-content'] > p ")

Answer 2

you could use:

libra_content = driver.find_elements(By.xpath, 'your_path')

read this:

how to find elements by xpath

Answer 3

Try This

<div>
    <h2 class="horoscope-content" >........</h2>
    <p class="horoscope-content" >........</p>            
    <p>.......</p>

libra_content = driver.find_elements(By.CLASS_NAME, 'horoscope-content')

Answer 4

libra_content = [x.find_element(By.XPATH,'./h2[1]').text for x in driver.find_elements(By.CLASS_NAME, 'horoscope-content')]

You could do something like this instead for both values if you want to store them both.

Answer 5

I solved it using css selectors, but didn't combine them into one. Another commenter's answer using xpath and class name combining the two is a possible solution.

libra_h2 = driver.find_element(By.CSS_SELECTOR, 'div.horoscope-content > h2')
libra_p = driver.find_element(By.CSS_SELECTOR, 'div.horoscope-content > p')