web 数据抓取问题我不知道如何从文件中导出信息。html 到我的 python 程序

Question

recently i start studying Web scraping, and today i made myself a challenge i tried to write information about every world from tibia.com, about what's name has world, how many people playing on it, what type of server is it, etc.

i created something like this

from urllib.request import urlopen, Request

from bs4 import BeautifulSoup as soup 

from fake_useragent import UserAgent


my_url = 'https://www.tibia.com/community/?subtopic=worlds'

uClient = urlopen(Request(my_url, headers={'User-Agent': 'Mozilla'}))


page_html = uClient.read()
uClient.close()

page_soup = soup(page_html, "html.parser")

containers = page_soup.findAll("tr", {"class":['Even', 'Odd']})

for container in containers:

    informations = containers.findAll("td")
    world = informations[0].txt

but i don't know how can I pull out information from td, my data file looks like:

<tr class="Odd">
<td style="width: 150px;"><a href="https://www.tibia.com/community/?subtopic=worlds&amp;world=Cosera">Cosera</a>
</td>
<td style="text-align: right;">75</td>
<td>North America</td>
<td>Optional PvP</td>

it's one from 92 worlds, and what i'm looking for is how can i extract information about world from this line

<td style="width: 150px;"><a href="https://www.tibia.com/community/?subtopic=worlds&amp;world=Cosera">Cosera</a>

and if you give me note how to do this, everything else i think i will figure out.

If someone has idea I would be greatful for your clue.

Answer 1

I'm not exactly sure what you mean but I'll try to give a solution to your problem.
It looks like you're trying to get all the row information from the table on the page. The simplest way to do this is to first get all the <tr> elements (all the rows) which you had already successfully done.
Then we want to loop through these rows to extract the data from them.
I'm not sure if you only want the 'Cosera' world, or just the whole table. If you want the whole table you can just remove the if statement in the code below.

from urllib.request import urlopen, Request

from bs4 import BeautifulSoup as soup 

my_url = 'https://www.tibia.com/community/?subtopic=worlds'

world_to_find = 'Cosera'

uClient = urlopen(Request(my_url, headers={'User-Agent': 'Mozilla'}))


page_html = uClient.read()
uClient.close()

page_soup = soup(page_html, "html.parser")

all_rows = page_soup.find_all('tr', {"class":["Odd", "Even"]})


for row in all_rows:
    if (row.select_one("td").text == world_to_find):
        data = {}
        
        row = row.findChildren("td" , recursive=False)
        data['world'] = row[0].text
        data['online'] = row[1].text
        data['location'] = row[2].text
        data['pvp_type'] = row[3].text
        data['additional_info'] = row[5].text
        
        print(data)

Outputs:

{'world': 'Cosera', 'online': '86', 'location': 'North America', 'pvp_type': 'Optional PvP', 'additional_info': 'blocked'}

If this wasn't what you meant please explain in your post what exactly you want the output to be