如何获取重复标签内的元素？

Question

我对 python 和 XML 解析很陌生。 我需要从 Internet 解析一个 XML 文件，我遇到了这个问题，我不知道如何获取我想要的信息，因为它在文档中使用了数百次的通用标签内。

XML 文件遵循以下结构：

<text>
  <dl>
    <dt>1. Information: </dt>
    <dd>
      <dl>
        <dt>1.1) Name: </dt>
        <dd>Company name.</dd>
        <dt>1.2) ID: </dt>
        <dd>Number.</dd>
        <dt>1.3) Address: </dt>
        <dd>San Bernardo, 45.</dd>
        <dt>1.4) City: </dt>
        <dd>Madrid.</dd>
        <dt>1.5) Province: </dt>
        <dd>Madrid.</dd>
        <dt>1.6) Postal Code: </dt>
        <dd>28015.</dd>
        <dt>1.7) Country: </dt>
        <dd>Spain.</dd>
        <dt>1.8) Code: </dt>
        <dd>587463.</dd>
        <dt>1.9) Phone: </dt>
        <dd>+34 PhoneNumber.</dd>
        <dt>1.11) email: </dt>
        <dd>example@gmail.com</dd>
        <dt>1.13) Buyer Address: </dt>
        <dd>https://example_url.com</dd>
      </dl>
    </dd>
    <dt>2. Organization:</dt>
    <dd>
      <dl>
        <dt>2.1) Type: </dt>
        <dd>Administration</dd>
        <dt>2.2) Activity: </dt>
        <dd>Administration.</dd>
      </dl>
    </dd>
    <dt>4. Codes:</dt>
    <dd>
      <dl>
        <dt>4.1) Main Code: </dt>
        <dd>72000000 (Services: software, internet), 72260000 (Software related services) and 72600000 (Informatic services).</dd>
        <dt>4.2) Code 2</dt>
        <dd>72000000 (Services: software, internet), 72260000 (Software related services) and 72600000 (Informatic services).</dd>
        <dt>4.3) Code 3</dt>
        <dd>72000000 (Services: software, internet), 72260000 (Software related services) and 72600000 (Informatic services).</dd>
        <dt>4.4) Code 4</dt>
        <dd>72000000 (Services: software, internet), 72260000 (Software related services) and 72600000 (Informatic services).</dd>
      </dl>
    </dd>
</text>

这只是文档中的一小部分。 从这里我想得到标签内的所有代码"Codes" ，在这种情况下 output 应该类似于： "72000000,72260000,72600000" 。

在此示例中，每个“dd”标签中的所有代码都相同，但由于这适用于我们从 web 检索的任何文件，因此我应该获取所有代码，然后消除重复的代码。

为了解析文档上的其他标签，我目前正在以这种方式使用 BeatifulSoup：

解析文档：

soup = get_soup_from_url(url, "lxml")

def find_text_by_tag(self, soup, tag):
   item = soup.find(tag)
   return item.text if item else ''

方法 get_soup_from_url() 正在从另一个文件中导入，代码如下：

from selenium import webdriver      # Selenium is a web testing library. It is used to automate browser activities.
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from bs4 import BeautifulSoup       # Beautiful Soup is a Python package for parsing HTML and XML documents. It creates parse trees that is helpful to extract the data easily.

def get_driver_from_url(url):
    options = webdriver.ChromeOptions()
    options.add_argument("--no-sandbox") 
    options.add_argument("--disable-dev-shm-usage") 
    options.add_argument("--headless") 
    driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=options)
    driver.get(url)
    return driver

def get_soup_from_url(url, parser):
    driver = get_driver_from_url(url)
    content = driver.page_source
    soup = BeautifulSoup(content, parser)
    return soup

def get_soup_from_driver(driver, parser):
    content = driver.page_source
    soup = BeautifulSoup(content, parser)
    return soup

def get_page_source_from_url(url):
    driver = get_driver_from_url(url)
    return driver.page_source

提前感谢任何给我有关如何执行此操作的信息的人

Answer 1

根据您的示例 HTML/XML select 您的元素与css selector ：

data = list(soup.select_one('dt:-soup-contains(" Codes:") + dd dl').stripped_strings)

然后迭代list以创建一个dict并使用@Sergey K 提到的模式仅提取数字：

{k.split(') ')[-1]:re.findall(r'\d{8}', v) for k, v in zip(data[::2], data[1::2])}

这导致：

{'Main Code:': ['72000000', '72260000', '72600000'],
 'Code 2': ['72000000', '72260000', '72600000'],
 'Code 3': ['72000000', '72260000', '72600000'],
 'Code 4': ['72000000', '72260000', '72600000']}

例子

from bs4 import BeautifulSoup

重新进口

html = '''
<text>
  <dl>
    <dt>1. Information: </dt>
    <dd>
      <dl>
        <dt>1.1) Name: </dt>
        <dd>Company name.</dd>
        <dt>1.2) ID: </dt>
        <dd>Number.</dd>
        <dt>1.3) Address: </dt>
        <dd>San Bernardo, 45.</dd>
        <dt>1.4) City: </dt>
        <dd>Madrid.</dd>
        <dt>1.5) Province: </dt>
        <dd>Madrid.</dd>
        <dt>1.6) Postal Code: </dt>
        <dd>28015.</dd>
        <dt>1.7) Country: </dt>
        <dd>Spain.</dd>
        <dt>1.8) Code: </dt>
        <dd>587463.</dd>
        <dt>1.9) Phone: </dt>
        <dd>+34 PhoneNumber.</dd>
        <dt>1.11) email: </dt>
        <dd>example@gmail.com</dd>
        <dt>1.13) Buyer Address: </dt>
        <dd>https://example_url.com</dd>
      </dl>
    </dd>
    <dt>2. Organization:</dt>
    <dd>
      <dl>
        <dt>2.1) Type: </dt>
        <dd>Administration</dd>
        <dt>2.2) Activity: </dt>
        <dd>Administration.</dd>
      </dl>
    </dd>
    <dt>4. Codes:</dt>
    <dd>
      <dl>
        <dt>4.1) Main Code: </dt>
        <dd>72000000 (Services: software, internet), 72260000 (Software related services) and 72600000 (Informatic services).</dd>
        <dt>4.2) Code 2</dt>
        <dd>72000000 (Services: software, internet), 72260000 (Software related services) and 72600000 (Informatic services).</dd>
        <dt>4.3) Code 3</dt>
        <dd>72000000 (Services: software, internet), 72260000 (Software related services) and 72600000 (Informatic services).</dd>
        <dt>4.4) Code 4</dt>
        <dd>72000000 (Services: software, internet), 72260000 (Software related services) and 72600000 (Informatic services).</dd>
      </dl>
    </dd>
</text>
'''
soup = BeautifulSoup(html)
data = list(soup.select_one('dt:-soup-contains(" Codes:") + dd dl').stripped_strings)
{k.split(') ')[-1]:v for k, v in zip(data[::2], data[1::2])}