Select an element in Selenium, Python, not by using XPath

Question

I was trying to scrape a website, and I need to select only the ul element inside the div with a class "Slider__SliderWrapper-sc-143uniy-0 jrPmnS", however, since there are many div tags with the same class, the only way I have to select just the ul I need is by looking at the href of the a tag, the one inside the h2. I can't use xpath, because div tags always change position.

<div>
   <h2><a class="slider-components__SectionLink-sc-1r2bduf-3 jchpWs" href="rightOne">Right!</a></h2>
   <div class="Slider__SliderWrapper-sc-143uniy-0 jrPmnS">
      <ul class="Slider__List-sc-143uniy-1 MTYOL">
      the right ul
      </ul>
   </div>
</div>
<div>
   <h2><a class="slider-components__SectionLink-sc-1r2bduf-3 jchpWs" href="wrongOne">Something else</a></h2>
   <div class="Slider__SliderWrapper-sc-143uniy-0 jrPmnS">
      <ul class="Slider__List-sc-143uniy-1 MTYOL">
      the wrong ul
      </ul>
   </div>
</div>

I thought about using css selector but I don't know how to, any help?

Answer 1

You definitely CAN use xpath to access the href attribute AND it's contents:

//a[contains(@href,'rightOne')]

and for the ul:

//h2/a[contains(@href,'rightOne')]/../following-sibling::div/ul

Answer 2

try xpath

//a[@href='rightOne']/../following-sibling::div/ul

Explanation:

You cannot use css_selector or any other locator since you are depending on a tag and you have to traverse upwards in DOM first, we are using /.. for that, alternatively you can use /parent::h2 and the next following-sibling using /following-sibling::div and then finally ul child

Answer 3

You cannot get a parent element with css selector, as it's not possible. Check here Is there a CSS parent selector?

In your case you would need to get the parent of a[href=rightOne] and get the ul of the following sibling.

With css you could use one of these locators:

div:nth-child(1) .Slider__SliderWrapper-sc-143uniy-0.jrPmnS>.Slider__List-sc-143uniy-1.MTYOL

Or

div:nth-child(1) .Slider__SliderWrapper-sc-143uniy-0.jrPmnS>ul

I would select any of XPaths proposed in other two answers if there are not restrictions on selectors.

But, if you are using such libraries as BeautfulSoup , you will have to use css selectors, as it does not support XPath. So, use the ones I proposed.