如何在网站上使用BeautifulSoup或Slimit从javascript变量输出电子邮件地址

Question

我有以下示例网站： http : //www.example.com/whatever.asp?profile=1

对于每个配置文件编号，此Java脚本代码中都有一封不同的电子邮件。

<script LANGUAGE="JavaScript">
function something()
{
var ptr;
ptr = "";
ptr += "<table><td class=france></td></table>";
ptr += "<table><td class=france><a href=mailto:exa";
ptr += "mple@email.com>email</a></td></table>";
document.all.something.innerHTML = ptr;
}
</script>

我想解析或正则表达式电子邮件地址。 电子邮件的位置取决于长度。 但是，只有此python代码可以解析mple@email.com，而不能解析example@email.com

url=urllib.urlopen('http://www.example.com/whatever.asp?profile=1')
contents= url.read()   
soup = BeautifulSoup(contents)
js_content= soup.findAll("script")[0].text
reg = '(<)?(\w+@\w+(?:\.\w+)+)(?(1)>)'
match= re.search(reg,js_content)
print match.group()

有什么帮助吗？ 谢谢。

Answer 1

#!/usr/bin/env python

from bs4 import BeautifulSoup
import re

soup = '''
<script LANGUAGE="JavaScript">
function something()
{
var ptr;
ptr = "";
ptr += "<table><td class=france></td></table>";
ptr += "<table><td class=france><a href=";
ptr += "mailto:example@knesset.com>email</a></td></table>";
document.all.something.innerHTML = ptr;
}
</script>
'''


soup = BeautifulSoup(soup)

for script in soup.find_all('script'):
    reg = '(<)?(\w+@\w+(?:\.\w+)+)(?(1)>)'
    reg2 = 'mailto:.*'
    secondHalf= re.search(reg, script.text)
    firstHalf= re.search(reg2, script.text)
    secondHalfEmail = secondHalf.group()
    firstHalfEmail = firstHalf.group()
    firstHalfEmail = firstHalfEmail.replace('mailto:', '')
    firstHalfEmail = firstHalfEmail.replace('";', '')
    if firstHalfEmail == secondHalfEmail:
        email = secondHalfEmail
    else:
        if ('>') not in firstHalfEmail:
            if ('>') not in secondHalfEmail:
                if firstHalfEmail != secondHalfEmail:
                    email = firstHalfEmail + secondHalfEmail
            else:
                email = firstHalfEmail
        else:
            email = secondHalfEmail

    print email

Answer 2

我建议您使用re.findall而不是re.search因为搜索将仅返回第一个匹配项。

url=urllib.urlopen('http://www.example.com/whatever.asp?profile=1')
contents= url.read()   
soup = BeautifulSoup(contents)
js_content= soup.findAll("script")[0].text
reg = r'<?(\w+@\w+(?:\.\w+)+)>?'
match= re.findall(reg,js_content)