BeautifulSoup Python 提取带有属性的特定标签的标签标题

Question

I'm working on a scraper using beautifulsoup to pull concert information for certain artists on songkick. the url I'm working with is here https://www.songkick.com/metro-areas/17835-us-los-angeles-la/february-2020?page=1 . I've been able to extract all artist, venue, city, and state info, the only thing I'm having trouble with is extracting the date of concerts.

In looking at the html elements, I see that the dates for shows are listed as the li title="Saturday 01 February 2020" values for example the children under ul class="event-listings". A method I was attempting to perform was extracting the time datetime values that are nensted under the li titles, but my output included the entire html markup for each li time datetime instead of just the datetime. I'm looking to either extract the li titles or the time datetime values. These li's don't have a class either.

Here is some of my code

import requests
from bs4 import BeautifulSoup as bs4

pages=[]
artists=[]
venues=[]
dates=[]
cities=[]
states=[]

pages_to_scrape=1

for i in range(1, pages_to_scrape+1):
    url = 'https://www.songkick.com/metro-areas/17835-us-los-angeles-la/february-2020?page={}'.format(i)
    pages.append(url)
for item in pages:
    page = requests.get(item)
    soup = bs4(page.text, 'html.parser')
    for m in soup.findAll('li', title=True):
        date = m.find('time')
        print(date)

Output:

<time datetime="2020-02-01T20:00:00-0800"></time>
<time datetime="2020-02-01T20:00:00-0800"></time>
<time datetime="2020-02-01T19:00:00-0800"></time>
<time datetime="2020-02-01T19:00:00-0800"></time>
<time datetime="2020-02-01T21:00:00-0800"></time>
etc...

Looking for output like this:

2020-02-01
2020-02-01
2020-02-01
etc...

Or if able to grab the title values of li's some how output like this:

Saturday 01 February 2020
Saturday 01 February 2020
Saturday 01 February 2020
Saturday 01 February 2020
etc...

I'm curious if I'm able to split at the " for the time datetime, but since it's not text I don't think that's possible. Also, I don't want to grab the first li class = "with-date" as that is just the headline for dates for the page as to why I'm not just grabbing all li's.

Answer 1

尝试m.find('time')['datetime']而不是m.find('time')

Answer 2

Here's a way to achieve this:

import requests
from bs4 import BeautifulSoup
page = requests.get("https://www.songkick.com/metro-areas/17835-us-los-angeles-la/february-2020?page=1")
soup = BeautifulSoup(p.content, "html.parser")
tags = soup.find_all("time")
[t["datetime"].split("T")[0] for t in tags]

Notes:

1. I'm quite sure that crawling Songkick in this way violates their terms and conditions.
2. You might consider using their API, which works well: https://www.songkick.com/developer