在Xpath中连接XML / HTML的后继文本节点

Question

I have this html fragment:

<tr>
    <th scope="row" style="text-align:left;">Appeared in</th>
    <td class="" style="">1972<sup id="cite_ref-dottcl_2_2-0" class="reference"><a href="#cite_note-dottcl_2-2"><span>[</span>2<span>]</span></a></sup></td>
</tr>
<tr>
    <th scope="row" style="text-align:left;">Usual 
<a href="/wiki/Filename_extension" title="Filename extension">filename extensions</a>
    </th>
    <td class="" style="">.h .c</td>

</tr>

I am using //th//text() expression to parse it.

The problem is it's returning ['Appeared in', 'Usual', 'filename extensions'] .

What I want is ['Appeared in', 'Usual filename extensions'] .

Answer 1

You need XPath 2.0 for doing this which is not supported by most XML-libraries of those scripting languages (including scrapy).

If you can use a more capable XPath processor (also have a look at XQuery 1.0 and newer, they all include at least XPath 2.0 as a subset), use:

//th/data()

/data() is equivalent to /data(.) which calls the function for the current context.

data() vs text()

While text() is not a function call, but node filter (and thus //text() is an axis step adding all text nodes individually to the result sequence), data() is a function aggregating all data for the current context (here: each <th/> individually).

XPath 1.0 limitations

There is no way to call any function that concatenates strings for each table header element individually: Function calls in axis steps are not supported, nor are explicit loops like they're possible in XPath 2.0.

Answer 2

Ah I will get downvoted for parsing HTML with regex , but can't be helped:

$html = '<tr>
    <th scope="row" style="text-align:left;">Appeared in</th>
    <td class="" style="">1972<sup id="cite_ref-dottcl_2_2-0" class="reference"><a href="#cite_note-dottcl_2-2"><span>[</span>2<span>]</span></a></sup></td>
</tr>
<tr>
    <th scope="row" style="text-align:left;">Usual 
<a href="/wiki/Filename_extension" title="Filename extension">filename extensions</a>
    </th>
    <td class="" style="">.h .c</td>

</tr>';

$html = str_replace("\r", '', str_replace("\n", '', $html)); // Remove new lines
preg_match_all('#<th[^>]*>(.*?)</th>#is', $html, $m); // Match what's between th tag

$result = array_map('strip_tags', $m[1]); // Get ride of html tags
print_r($result);// printing the results

Output:

Array
(
    [0] => Appeared in
    [1] => Usual filename extensions    
)