拼抢 <title>用lxml的iterparse标记

Question

I'm running into a problem with using lxml's iterparse on my HTML. 我在HTML上使用lxml的iterparse了问题。 I'm trying to get the <title> 's text but this simple function doesn't work on complete web pages: 我正在尝试获取<title>的文本，但这个简单的函数不适用于完整的网页：

def get_title(str):
    titleIter = etree.iterparse(StringIO(str), tag="title")
    try:
        for event, element in titleIter:
            return element.text
        # print "Script goes here when it doesn't work"
    except etree.XMLSyntaxError:
        return None

This function works fine on simple input like " <title>test</title> ", but when I give it a complete page it's unable to extract the title. 这个函数在“ <title>test</title> ”之类的简单输入上工作正常，但是当我给它一个完整的页面时，它无法提取标题。

UPDATE: Here's the HTML I'm working with: 更新：这是我正在使用的HTML：

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html dir="ltr" lang="it" xmlns="http://www.w3.org/1999/xhtml">
<head>
<link rel="icon" href="http://www.tricommerce.it/tricommerce.ico" />
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
<title>Tricommerce - Informazioni sulla privacy</title>
<meta name="description" content="Info sulla privacy" />
<meta name="keywords" content="Accessori notebook Alimentatori Case Cavi e replicatori Controllo ventole Lettori e masterizzatori Modding Pannelli &amp; display Dissipatori Tastiere e mouse Ventole Griglie e filtri Hardware Accessori vari Box esterni Casse e cuffie Sistemi a liquido Paste termiche vendita modding thermaltake vantec vantecusa sunmbeam sunbeamtech overclock thermalright xmod aerocool arctic cooling arctic silver zalman colorsit colors-it sharkoon mitron acmecom Info sulla privacy" />
<meta name="robots" content="index, follow" />
<link rel="stylesheet" href="http://www.tricommerce.it/css/tricommerce.css" />
<link rel="stylesheet" href="css/static.css" />
<script type="text/javascript" src="http://www.tricommerce.it/javascript/vertical_scroll.js"></script>

<script type="text/javascript">
//<![CDATA[
function MM_preloadImages() { //v3.0
 var d=document; if(d.images){ if(!d.MM_p) d.MM_p=new Array();
   var i,j=d.MM_p.length,a=MM_preloadImages.arguments; for(i=0; i<a.length; i++)
   if (a[i].indexOf("#")!=0){ d.MM_p[j]=new Image; d.MM_p[j++].src=a[i];}}
}
//]]>
</script>

<link rel="stylesheet" type="text/css" href="http://www.tricommerce.it/css/chromestyle.css" />

<script type="text/javascript" src="http://www.tricommerce.it/javascript/chrome.js">
/***********************************************
* AnyLink CSS Menu script- ? Dynamic Drive DHTML code library (www.dynamicdrive.com)
* This notice MUST stay intact for legal use
* Visit Dynamic Drive at http://www.dynamicdrive.com/ for full source code
***********************************************/
</script>

</head>
</html>

Also, a quick note on why I'm using iterparse-- it's because I don't want to load in the entire DOM just to get a single tag early on in the document. 另外，快速说明我为什么使用iterparse--这是因为我不想加载整个DOM只是为了在文档的早期获得单个标记。

Answer 1

You might want to post at least a portion of the data you're actually trying to parse. 您可能希望发布至少部分您实际尝试解析的数据。 Absent that information, here's a guess. 没有这些信息，这是一个猜测。 If the <html> element defines a default XML namespace, you'll need to use that when looking for elements. 如果<html>元素定义了默认的XML命名空间，则在查找元素时需要使用它。 For example, look at this simple document: 例如，看看这个简单的文档：

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.w3.org/MarkUp/SCHEMA/xhtml11.xsd"
  xml:lang="en">
  <head>            
    <title>Document Title</title>
  </head>
  <body>
  </body>
</html>

Given this input, the following will return no results: 鉴于此输入，以下将不返回任何结果：

>>> doc = etree.parse(open('foo.html'))
>>> doc.xpath('//title')
[]

This fails because we're look for a <title> element without specifying a namespace...and absent a namespace, the parser isn't going to find a match (because foo:title is different from bar:title , assuming that foo: and bar: are defined XML namespaces). 这失败是因为我们在没有指定命名空间的情况下寻找<title>元素...并且没有命名空间，解析器不会找到匹配项（因为foo:title与bar:title不同，假设foo:和bar:是定义的XML命名空间）。

You can explicitly use a namespace with the ElementTree interface like this: 您可以使用ElementTree接口显式使用名称空间，如下所示：

>>> doc.xpath('//html:title',
...   namespaces={'html': 'http://www.w3.org/1999/xhtml'})
[<Element {http://www.w3.org/1999/xhtml}title at 0x1087910>]

And there's our match. 这是我们的比赛。

You can pass namespace prefixes to the tag argument of iterparse, too: 您也可以将名称空间前缀传递给iterparse的tag参数：

>>> titleIter = etree.iterparse(StringIO(str), 
...   tag='{http://www.w3.org/1999/xhtml}title')
>>> list(titleIter)
[(u'end', <Element {http://www.w3.org/1999/xhtml}title at 0x7fddb7c4b8c0>)]

If this doesn't solve your problem, post some sample input and we'll work from there. 如果这不能解决您的问题，请发布一些示例输入，我们将从那里开始工作。

拼抢 <title>用lxml的iterparse标记

问题描述

1 个解决方案

解决方案1
2 已采纳 2012-04-24 01:40:07

拼抢 <title>用lxml的iterparse标记

问题描述

1 个解决方案

解决方案1 2 已采纳 2012-04-24 01:40:07

解决方案1
2 已采纳 2012-04-24 01:40:07