在美丽的汤刮文本与文本

Question

I am trying to scrape a page using python and beautiful soup bs4

I want to keep the text in the <p> element in the page along with the emojis in this text.

The first attempt was:

import urllib
import urllib.request
from bs4 import BeautifulSoup

urlobject = urllib.request.urlopen("https://example.com")

soup = BeautifulSoup(urlobject, "lxml")

result= list(map(lambda e: e.getText(), soup.find_all("p", {"class": "text"})))

But this doesn't include emojis. I then tried to remove .getText() and just keep :

result= list(map(lambda e: e, soup.find_all("p", {"class": "text"})))

Which made me realize the emojis in this website are in the alt of img tags:

<p class="text">I love the night<img alt="🌟" class="emoji" src="etc"/><span>!</span></p>

So what I want to do is :

• getText() for p with class text
• But also get alt for img with class=emoji

And keep the text and the emojis as one sentence.

Is there any way to do this?

Any help would be appreciated.

Answer 1

How about the following, returning tuples of the targeted data for each p ? I just used your example p element twice as the input for this test:

from bs4 import BeautifulSoup

s = """
<p class="text">I love the night<img alt="🌟" class="emoji" src="etc"/><span>!</span></p>
<p class="text">I love the night<img alt="🌟" class="emoji" src="etc"/><span>!</span></p>
"""

soup = BeautifulSoup(s, 'lxml')

elements = soup.find_all('p', {'class': 'text'})
print(list(map(lambda e: (e.getText(), e.find('img', {'class': 'emoji'})['alt']), elements)))

Result:

[('I love the night!', '🌟'), ('I love the night!', '🌟')]

Answer 2

if the img.emoji are optional you can try below, and it will preserve emoji position

urlobject = '''<p class="text">I love the night<img alt="🌟" class="emoji" src="etc"/><span>!</span></p>
<p class="text">I love the day<span>!</span></p>
<p class="text">I love the music<img alt="🌟" class="emoji" src="etc"/> <img alt="🔊" class="emoji" src="etc"/><span>!</span></p>
'''

result = []
for p in soup.find_all('p', {'class': 'text'}):
    emoji = p.select('img.emoji')
    if emoji:
        for em in emoji:
            index = p.contents.index(em)
            p.contents[index].replace_with(em['alt'])
    result.append(p.getText())

print(result)

Results:

['I love the night🌟!', 'I love the day!', 'I love the music🌟 🔊!']