如何在忽略某個標簽的同時從 Python 中的 html 中提取文本
[英]How to extract text from html in Python while ignoring a certain tag
問題描述
我有一個如下所示的輸入文件，我正在嘗試提取其文本並刪除 html 標簽。 請注意，我希望每個 p 都在換行符中，但如果它是 br，我想將它保留在同一行中，但無論如何都刪除 br 標簽。
<tt xmlns="http://www.w3.org/ns/ttml" xml:lang="en" xmlns:tts="http://www.w3.org/ns/ttml#parameter"><head><styling><style id="b1"/></sty ling></head><body><div xml:lang="en" style="b1"><p begin="" end="0.143">HISTORY</p><p begin="0.143" end="0.286">HISTORY TV"</p><p begin= "0.286" end="0.714">HISTORY TV" THIS</p><p begin="0.714" end="0.857">HISTORY TV" THIS<br/>WEEKEND</p><p begin="0.857" end="3">HISTORY TV " THIS<br/>WEEKEND ON</p><p begin="3" end="3.333">HISTORY TV" THIS<br/>WEEKEND ON C-SPAN3.</p><p begin="3.333" end="3.667">WEEKEND ON C- SPAN3.<br/>>>></p><p begin="3.667" end="4">WEEKEND ON C-SPAN3.<br/>>>> "THE</p><p begin="4" end="4.5">WEEKEND ON C-SPA N3.<br/>>>> "THE MARCH</p><p begin="4.5" end="5">WEEKEND ON C-SPAN3.<br/>>>> "THE MARCH ON</p><p begin="5" end="5.5">W EEKEND ON C-SPAN3.<br/>>>> "THE MARCH ON WASHINGTON"</p><p begin="5.5" end="5.667">>>> "THE MARCH ON WASHINGTON"<br/>F OR</p><p begin="5.667" end="5.833">>>> "THE MARCH ON WASHINGTON"<br/>FOR JOBS</p><p begin="5.833" end="6">>>> "THE MAR CH ON WASHINGTON"<br/>FOR JOBS AND</p><p begin="6" end="6.2">>>> "THE MARCH ON WASHINGTON"<br/>FOR JOBS AND FREEDOM</p><p begin ="6.2" end="6.4">>>> "THE MARCH ON WASHINGTON"<br/>FOR JOBS AND FREEDOM WAS</p><p begin="6.4" end="7">>>> "THE MARCH O N WASHINGTON"<br/>FOR JOBS AND FREEDOM WAS 49</p><p begin="7" end="8">FOR JOBS AND FREEDOM WAS 49<br/>YEARS</p><p begin="8" end="8.5">FO R JOBS AND FREEDOM WAS 49<br/>YEARS AGO.</p><p begin="8.5" end="8.75">YEARS AGO.<br/>ON</p><p begin="8.75" end="9">YEARS AGO.<br/>ON AUG UST</p><p begin="9" end="13">YEARS AGO.<br/>ON AUGUST 28th,</p><p begin="13" end="13.333">YEARS AGO.<br/>ON AUGUST 28th, 1963.</p><p beg in="13.333" end="13.5">ON AUGUST 28th, 1963.<br/>THE</p><p begin="13.5" end="13.667">ON AUGUST 28th, 1963.<br/>THE MARCH</p><p begin="13 .667" end="13.833">ON AUGUST 28th, 1963.<br/>THE MARCH WAS</p><p begin="13.833" end="14">ON AUGUST 28th, 1963.<br/>THE MARCH WAS ORKGANI ZED</p><p begin="14" end="14.167">ON AUGUST 28th, 1963.<br/>THE MARCH WAS ORKGANIZED TO</p><p begin="14.167" end="14.667">ON AUGUST 28th , 1963.<br/>THE MARCH WAS ORKGANIZED TO PUSH</p><p begin="14.667" end="14.833">THE MARCH WAS ORKGANIZED TO PUSH<br/>FOR</p>
所以最后我想要
HISTORY
HISTORY TV"
HISTORY TV" THIS
HISTORY TV" THIS WEEKEND
HISTORY TV" THIS WEEKEND ON
HISTORY TV" THIS WEEKEND ON C-SPAN3.
...etc
我如何完成這個任務？
我用了這個代碼
import re
import os
def remove_html_tags(data):
p = re.compile(r'<.*?>')
return p.sub(' ', str(data)).strip()
directory = './reprocess'
for filename in os.listdir(directory):
if filename.endswith(".dfxp"):
print("Processing: {}".format(filename))
with open("./reprocess/"+filename, "r") as inputFile:
data = inputFile.read().splitlines()
new_data = ""
for line in data:
new_data = new_data + remove_html_tags(line) + "\n"
with open("./rmout/"+filename, "w") as text_file:
text_file.write(new_data)
但它給了我一個可怕的輸出
HISTORY TV"
WEEKEND ON
HISTORY TV" THIS
"THE MARCH
"THE MARCH ON
WEEKEND ON C-SPAN3.
FOR JOBS AND FREEDOM WAS
"THE MARCH ON WASHINGTON"
YEARS
FOR JOBS AND FREEDOM WAS 49
ON AUGUST 28th,
YEARS AGO.
THE MARCH WAS ORKGANIZED TO
ON AUGUST 28th, 1963.
FOR COMPREHENSIVE CIVIL
THE MARCH WAS ORKGANIZED TO PUSH
INCLUDING PUBLIC
FOR COMPREHENSIVE CIVIL RIGHTS
DESEGREGATION,
DESEGREGATION, VOTING
2 個解決方案
解決方案1
0 2020-10-06 08:26:12
消化
代碼使用bs4 （BeautifulSoup4， official docs ），包括3個主要步驟：
- 前湯數據清理：有時清理原始文本中的某些數據比清理湯中的數據更方便。 如果是這種情況，請不要猶豫。
- 構建湯（DOM）
- 湯元素提取並對提取的文本進行后處理。
代碼
免責聲明：廣泛測試並始終期待例外。 問題解決者無法預見未出現在樣本數據中的問題。
import bs4
import re
from pprint import pprint
# raw data
html = "(as provided)"
# 1. cleansing
# (1) remove known unwanted patterns
html = html.replace(" ", "")
html = html.replace(">>>", "")
# remove <br> tags (can also remove after the soup is built)
html = re.sub(r"<br\s*/?>", " ", html) # careful! error-prone!
# (2) regularize multiple spaces
html = re.sub(r"\s{2,}", " ", html)
# 2. construct soup (DOM)
soup = bs4.BeautifulSoup(html, 'html.parser')
# 3. extract text in target elements
ls_lines = []
for el in soup.find_all("p"):
ls_lines.append(el.get_text().strip())
# check
for line in ls_lines:
print(line)
輸出
輸出現在看起來非常不錯。 但是，這是因為在小樣本數據集中找不到太多問題。 在實際情況下，可能需要更多的預處理和元素選擇規則。 這部分超出了這個問題的范圍。
HISTORY
HISTORY TV"
HISTORY TV" THIS
HISTORY TV" THIS WEEKEND
HISTORY TV" THIS WEEKEND ON
HISTORY TV" THIS WEEKEND ON C-SPAN3.
WEEKEND ON C-SPAN3.
WEEKEND ON C-SPAN3. "THE
WEEKEND ON C-SPAN3. "THE MARCH
WEEKEND ON C-SPAN3. "THE MARCH ON
WEEKEND ON C-SPAN3. "THE MARCH ON WASHINGTON"
"THE MARCH ON WASHINGTON" FOR
"THE MARCH ON WASHINGTON" FOR JOBS
"THE MARCH ON WASHINGTON" FOR JOBS AND
"THE MARCH ON WASHINGTON" FOR JOBS AND FREEDOM
"THE MARCH ON WASHINGTON" FOR JOBS AND FREEDOM WAS
"THE MARCH ON WASHINGTON" FOR JOBS AND FREEDOM WAS 49
FOR JOBS AND FREEDOM WAS 49 YEARS
FOR JOBS AND FREEDOM WAS 49 YEARS AGO.
YEARS AGO. ON
YEARS AGO. ON AUGUST
YEARS AGO. ON AUGUST 28th,
YEARS AGO. ON AUGUST 28th, 1963.
ON AUGUST 28th, 1963. THE
ON AUGUST 28th, 1963. THE MARCH
ON AUGUST 28th, 1963. THE MARCH WAS
ON AUGUST 28th, 1963. THE MARCH WAS ORKGANIZED
ON AUGUST 28th, 1963. THE MARCH WAS ORKGANIZED TO
ON AUGUST 28th, 1963. THE MARCH WAS ORKGANIZED TO PUSH
THE MARCH WAS ORKGANIZED TO PUSH FOR
參考
- Python - 使用 beautifulSoup 查找文本，然后替換原始湯變量
- 替換
<br>：在文本中； 在湯里。 - （一個 Kaggle 教程） 使用 BeautifulSoup 刪除 HTML 標簽
解決方案2
0 2021-02-08 16:45:59
輸入總是TTML嗎？ 如果是這樣， ttconv可以將 TTML/IMSC 文檔拆分為一系列中間同步文檔 (ISD)，每個 ISD 對應於 TTML/IMSC 文檔內容靜態的時間段。 可以輕松地從每個 ISD 中提取文本。
import ttconv.imsc.reader
import ttconv.isd
import xml.etree.ElementTree as et
tt_doc = """<?xml version="1.0" encoding="UTF-8"?>
<tt xml:lang="fr" xmlns="http://www.w3.org/ns/ttml">
<body>
<div>
<p begin="1s" end="2s">Hello</p>
<p begin="3s" end="4s">Bonjour</p>
</div>
</body>
</tt>"""
m = ttconv.imsc.reader.to_model(et.ElementTree(et.fromstring(tt_doc)))
st = ttconv.isd.ISD.significant_times(m)
for t in st:
isd = ttconv.isd.ISD.from_model(m, t)
# walk through all Text elements in `isd` to extract text
ttconv 還支持從 TTML/IMSC 轉換為 SRT，這是一種簡單的基於文本的格式。
tt.py convert -i <input .ttml file> -o <output .srt file> --otype SRT --itype TTML
如果其他HTML位於標記中，如何從div標記中提取python中的文本？
[英]How to extract text in python from div tag if other html is within the tag?
使用Python和Selenium，如何從包含以下內容的HTML中提取文本： <p> 標簽？
[英]Using Python & Selenium, how to extract the text from HTML containing the <p> tag?
在檢查粗體時從 HTML 文件中提取所有文本（Python）
[英]Extract all text from HTML file while checking for boldness (Python)
聲明:本站的技術帖子網頁，遵循CC BY-SA 4.0協議，如果您需要轉載，請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.