使用漂亮的汤选择 div 标签内的链接

Question

I am trying to run the following code

           headers = {
                'User-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 
                  (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36'
                     }

           params = {
                'q': 'Machine learning,
                'hl': 'en'
                    }
           html = requests.get('https://scholar.google.com/scholar', headers=headers, 
           params=params).text
           soup = BeautifulSoup(html, 'lxml')
           for result in soup.select('.gs_r.gs_or.gs_scl'):
             profiles=result.select('.gs_a a')['href']

The following output (error) is being shown "TypeError: list indices must be integers or slices, not str" What is it I am doing wrong?

Answer 1

The following is tested and works:

import requests
from bs4 import BeautifulSoup as bs

headers = {
'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.79 Safari/537.36'
}

params = {
    'q': 'Machine learning',
    'hl': 'en'
        }
html = requests.get('https://scholar.google.com/scholar', headers=headers, 
params=params).text
soup = bs(html, 'lxml')
for result in soup.select('.gs_r.gs_or.gs_scl'):
    profiles=result.select('.gs_a a')
    for p in profiles:
        print(p.get('href'))

Result in terminal:

/citations?user=rSVIHasAAAAJ&hl=en&oi=sra
/citations?user=MnfzuPYAAAAJ&hl=en&oi=sra
/citations?user=09kJn28AAAAJ&hl=en&oi=sra
/citations?user=yxUduqMAAAAJ&hl=en&oi=sra
/citations?user=MnfzuPYAAAAJ&hl=en&oi=sra
/citations?user=9Vdfc2sAAAAJ&hl=en&oi=sra
/citations?user=lXYKgiYAAAAJ&hl=en&oi=sra
/citations?user=xzss3t0AAAAJ&hl=en&oi=sra
/citations?user=BFdcm_gAAAAJ&hl=en&oi=sra
/citations?user=okf5bmQAAAAJ&hl=en&oi=sra
/citations?user=09kJn28AAAAJ&hl=en&oi=sra

In your code, you were trying to obtain the href attribute from a list ( soup.select returns a list, and soup.select_one return a single element).

See BeautifulSoup documentation here