Python BeautifulSoup extract text from SPAN and A tags

Question

I want extract text from SPAN and A tags and put in list like this schema:

['Farina', '500 g']['Uova', '1']['Sale','100 g']

scraping with BeautifulSoup

from bs4 import BeautifulSoup
import re
import string

markup = """
<dd class="ingredient">
    <a href="#">Farina</a>
    <span>500 g</span>
</dd>
<dd class="ingredient">
    <a href="#">Uova</a>
    <span>1</span>
</dd>
<dd class="ingredient">
    <a href="#">Sale</a>
    <span>100 g</span>
</dd>
"""

soup = BeautifulSoup(markup, 'html.parser')

allIngredients = []
for tag in soup.find_all(attrs={'class' : 'ingredient'}):
    #[tag.text for tag in tags]
    link = tag.a.get('href')
    nameIngredient = tag.a.string

    contents = tag.span.text
    quantityIngredient = re.sub(r"\s+", " ", contents).strip()
    allIngredients.append([nameIngredient, quantityIngredient])

print(allIngredients)

sometimes SPAN can be empty or not exist

Answer 1

Here is a solution using lxml (instead of bs4 )

from lxml import html

markup = """
<dd class="ingredient">
    <a href="#">Farina</a>
    <span>500 g</span>
</dd>
<dd class="ingredient">
    <a href="#">Uova</a>
    <span>1</span>
</dd>
<dd class="ingredient">
    <a href="#">Sale</a>
    <span>100 g</span>
</dd>
<dd class="ingredient">
    <a href="#">Vino</a>
</dd>
"""

root = html.fromstring(markup)
result = []
for node in root.xpath(".//dd"):
    a = node.xpath(".//a")
    span = node.xpath(".//span")
    result.append((
        a[0].text_content() if a else None, 
        span[0].text_content() if span else None
    ))


print(result)
# [('Farina', '500 g'), ('Uova', '1'), ('Sale', '100 g'), ('Vino', None)]