How do I get attribute for non tagged text inside div

Question

I am currently working on a script using python(and selenium) and having some issues when I try to get the text (III. AIR COMBAT) inside this div:

<div class="vs901-4">
<i id="copyarticle" style="cursor:pointer; color:white;margin-right:10px;" class="fa fa-copy"></i>
<span id="copiednotif" class="badge badge-pills badge-success" style="text-weight:300;cursor:pointer; margin-left: 5px;margin-right:5px;"></span>
<span id="profileid" class="hidden"> website link</span>
                             III. AIR COMBAT</div>

so basically I tried the usual full xpath way:

self.driver.find_element_by_xpath("/html/body/div[2]/div/div[2]/div[2]/div/div/div/div/div/div[1]/div[4]/text()").get_attribute("innerHTML") 

that's basically what I've been using for the other texts that I needed (the others weren't inside a non tagged div) and they all worked, but this one is giving me this error:

"The result of the xpath expression "/html/body/div[2]/div/div[2]/div[2]/div/div/div/div/div/div[1]/div[4]/text()" is: [object Text]. It should be an element."

Thanks for the help.

Answer 1

The error looks to be in the /text() of the xpath, you are already getting the text in there and not the element, to then get the innerHTML. Try with the following:

self.driver.find_element_by_xpath("/html/body/div[2]/div/div[2]/div[2]/div/div/div/div/div/div[1]/div[4]").get_attribute("innerHTML") 

or

self.driver.find_element_by_xpath("/html/body/div[2]/div/div[2]/div[2]/div/div/div/div/div/div[1]/div[4]/text()")

Answer 2

Here is the line of code that you can use to return text directly using javascript.

def get_text_exclude_children(element):
    return driver.execute_script(
        """
        var parent = arguments[0];
        var child = parent.firstChild;
        var textValue = "";
        while(child) {
            if (child.nodeType === Node.TEXT_NODE)
                textValue += child.textContent;
                child = child.nextSibling;
        }
        return textValue;""",
        element).strip() 

Now you can use the method as shown below.

element = driver.find_element_by_xpath("//div[@class='vs901-4']")
elementOnlyText = get_text_exclude_children(element)
print(elementOnlyText)
```