漂亮的汤网抓取返回 None-Python

Question

I have a list of movies that I want to scrap the genres from Google. I've built this code:

import requests
from bs4 import BeautifulSoup

list=['Se7en','Cinema Paradiso','The Shining','Toy Story 3','Capernaum']
gen2 = {}
for i in list:
  user_query = i +'movie genre'
  URL = 'https://www.google.co.in/search?q=' + user_query
  headers = {'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.5005.63 Safari/537.36'}
  page = requests.get(URL, headers=headers)
  soup = BeautifulSoup(page.content, 'html.parser')
  c = soup.find(class_='EDblX DAVP1')
  print(c)
  if c != None:
    genres = c.findAll('a')
    gen2[i]= genres

But it returns an empty dict, so I checked one by one and it worked, for example:

import requests
from bs4 import BeautifulSoup

user_query = 'Se7en movie genre' 
URL = "https://www.google.co.in/search?q=" + user_query
headers = {'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.5005.63 Safari/537.36'}
page = requests.get(URL, headers=headers)
soup = BeautifulSoup(page.content, 'html.parser')
v = soup.find(class_='KKHQ8c')
h = {}
genres = v.findAll('a')
for genre in genres:
  h['Se7en']=genre

So I find out that in the for loop the variable c is returning None. I can't figure out why! It only return None inside the loop.

Answer 1

Currently, your URLs are of the form URLs

so the returned results(google) aren't accurate for all the movies. You can change it to

`for i in list:
  i="+".join(i.split(" "));          
  user_query = i + "+movie+genre"
  URL = 'https://www.google.com/search?q=+'+user_query`

also, movies that belong to a single genre like Cinema Paradiso are in a div with class name "Z0LcW".