如何检测网站上的变化？ python web 刮

Question

I started doing a monitor python for a shoe website. Now I would like to know if there is a way to know when the site is updated. For example: if there is a change in the available shoe sizes -> Send webhook to my discord

I don't know how to detect changes on the site.. please help me. If you have an idea, let me to know:)

img webhook discord

from dhooks import Webhook, Embed
import requests
import bs4
from bs4 import BeautifulSoup
import lxml

url = "https://en.aw-lab.com/women/shoes/new-arrivals-AW_10008AAQB.html?cgid=women_shoes_newin&dwvar_AW__10008AAQB_color=5011614"

headers = {
    'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36'}

res = requests.get(url, headers=headers)
res.raise_for_status()
soup = BeautifulSoup(res.text, "lxml")
img_shoes = "https://en.aw-lab.com/dw/image/v2/BCLG_PRD/on/demandware.static/-/Sites-awlab-master-catalog/default/dwd9415a8e/images/large/5011614_0.jpg?sw=843"
size = soup.select(".b-size-selector__item-0")
array_size = []

url_shoes = "[ADIDAS SUPERSTAR BOLD](" + url + ")"

embed = Embed(
    description=url_shoes,
    color=0x5CDBF0,
    timestamp='now'
)

for sizes in size:
    get_sizes = sizes.getText()
    array_size.append(get_sizes.strip())


embed.add_field(name="Size", value=('\n'.join(map(str, array_size))))

embed.set_thumbnail(img_shoes)

hook.send(embed=embed)

Answer 1

You can use the hashlib module to compute a checksum of the page, save it and then compute it again to check if it changed. NOTE: any subtle change will change the checksum!

import hashlib

# ...

checksum = hashlib.sha256(res.text.encode('utf-8')).hexdigest()

# save it to a txt file as a comparison for the next accesses