我将如何使用美丽的汤从这个网站上抓取数据？

Question

[![Problem][1]][1]

Above is the HTML, what the website looks like, and my code. I am trying to extract this information into a dictionary. for example {"Official Symbol: ELF4"} and so on. I have already watched a few tutorials but I'm still confused. can anyone help me out?

import requests
from bs4 import BeautifulSoup
url = "https://www.ncbi.nlm.nih.gov/gene/2000"
r  = requests.get(url)
data = r.content
soup = BeautifulSoup(data, 'html.parser')
#text_found = soup.find("dd",attrs={"class":"noline"}).text

dd_data = soup.find_all("dd")
for dditem in dd_data:
    if dditem != "None":
        print(dditem.string)

da_data = soup.find_all("dt")
for daitem in da_data:
    if daitem != "None":
        print(daitem.string)

Answer 1

To scrape the data as a dict see the following example:

import requests
from bs4 import BeautifulSoup


URL = "https://www.ncbi.nlm.nih.gov/gene/2000"
soup = BeautifulSoup(requests.get(URL).content, "html.parser")

result = {
    k.text.replace(" ", "").replace("\n", " "): v.find_next(text=True)
    for k in soup.select("dt.noline")
    for v in soup.select("dd.noline")
}


print(result)

Output:

{'Official Symbol': 'ELF4'}

Answer 2

I think you can just create two lists, fill them in your loops or at once like this:

dd_data = soup.find("dl", { "id" : "summaryDl" })


fields =[]
contents = []

# Append them sequentially, assuming the order is correct
for dditem in dd_data.find_all():
    if dditem.name == "dd":
        fields.append(dditem.text)
    if dditem.name == "dt":
        contents.append(dditem.text)
        
# zip the two lists together creating a list of pairs, then make a dictionary out of the list of pairs
contents = dict((zip(contents,fields)))