抓取时提取列表值

Question

I'm looking through https://www.nps.gov/index.htm and trying to create a dictionary where the state names from the drop-down menu are the keys and the values are the links to the appropriate page containing that state's information.

However, with my current code, I am getting something like this:

<li><a href="/state/wy/index.htm">Wyoming</a></li>

With my current skill level I don't know how to extract the state name, because it doesn't have any identifier or class or anything right?

So how would I go about achieving this? Here is my current code:

state_dict = {}

url = 'https://www.nps.gov/index.htm'
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')
drop_down_search = soup.find('ul', class_="dropdown-menu SearchBar-keywordSearch")
state_search = drop_down_search.find_all('li', recursive=True)

for state in state_search:
    print(state)

Answer 1

You can use .text property, just like this:

import requests
from bs4 import BeautifulSoup

state_dict = {}

url = 'https://www.nps.gov/index.htm'
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')
drop_down_search = soup.find('ul', class_="dropdown-menu SearchBar-keywordSearch")
state_search = drop_down_search.find_all('li', recursive=True)

for state in state_search:
    print(state.text)

it will print only the text:

Alabama
Alaska
American Samoa
Arizona
Arkansas
...

Answer 2

...

for state in state_search:
    for link in state.find_all('a'):
        print("%30s ===> %s" % (link.text, link.get('href')))