I have following HTML file and I am trying to scrape the complete sentence using BeautifulSoup but couldn't get it. Currently I am getting only highlighted words. my desired output should be
Antenna booster has stopped sending signal files ,possible user network issue or BOOSTER issue.
Any solution?
</table>
<!--Record Header End-->
<span style="BACKGROUND-COLOR: #0000ff; color: #ffffff">
Antenna
</span>
<span style="BACKGROUND-COLOR: #0000ff; color: #ffffff">
booster
</span>
has stopped
<span style="BACKGROUND-COLOR: #0000ff; color: #ffffff">
sending
</span>
signal files ,possible user
<span style="BACKGROUND-COLOR: #0000ff; color: #ffffff">
network
</span>
<span style="BACKGROUND-COLOR: #ff0000">
issue
</span>
or BOOSTER
<span style="BACKGROUND-COLOR: #ff0000">
issue
</span>
.
<br>
<br>
<br>
Here is what I tried:
issue_field = soup.find_all('span', {'style':'BACKGROUND-COLOR: #0000ff; color: #ffffff'})
issue_str = str(issue_field)
Issue_corpora = [word.lower() for word in BeautifulSoup(issue_str,'html.parser').get_text().strip().split(',')]
print(Issue_corpora)
The problem is there are texts outside the elements. There is a duplicate question on SO already: Get text outside known element beautifulsoup
So here is the solution, maybe needs a little polishing. (note that variable t
contains the html as text)
from bs4 import BeautifulSoup as bs
soup = bs(t)
text = ''
for span in soup.findAll('span'):
text += getattr(span, 'text', '').strip() + ' '
text += getattr(span, 'nextSibling', '').strip() + ' '
Result using this approach is:
>>> In : text
>>> Out: u'Antenna booster has stopped sending signal files ,possible user network issue or BOOSTER issue . '
You can replace doubled spaces with a single space or remove space before comma or define rules to handle it while looping through span
elements.
from bs4 import BeautifulSoup
import re
example = """your example"""
soup = BeautifulSoup(example, "html.parser")
_text = ""
for span in soup.find_all('span', style=re.compile('BACKGROUND-COLOR:')):
_text += "%s %s" % (span.get_text(strip=True), span.next_sibling.replace("\n", ""))
print (re.sub(" +"," ", _text))
Use re
in the end to trim extra spaces.
Outputs:
Antenna booster has stopped sending signal files ,possible user network issue or BOOSTER issue .
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.