我可以從此 XHR 請求中提取 JSON 數據嗎？

Question

我是一個初學者，試圖在本網站https://www.csgoroll.com/en/withdraw/csgo/p2p的部分上進行網絡爬蟲，我正在嘗試提取名為“graphql”的請求中每個項目的價格和名稱?operationName=TradeList&variables=" soley 使用 Python 中的請求庫，但我不確定該怎么做。 我對此進行了一些研究，它使我使用了一個名為 Postman 的應用程序，如果我將 cURL 復制到它，它不會返回 JSON 數據。 進一步研究它看起來像它使用 graphql 但是當我連接到https://api.csgoroll.com/我認為它用來查詢數據的操場說服務器無法訪問。 所以我想知道是否可以僅使用 Python 中的請求庫來提取數據，如果可以的話如何？

Answer 1

它需要來自真正的 web 瀏覽器的 header User-Agent或至少短'Mozilla/5.0' 。

requests默認使用python/3.x之類的東西，因此服務器要求解析Captcha 。

import requests

url = 'https://api.csgoroll.com/graphql'

headers = {
    'User-Agent': 'Mozilla/5.0',
#    'Accept': 'application/json, text/plain, */*',
}

params = {
    'operationName': 'TradeList',
    'variables': '{"first":50,"orderBy":"TOTAL_VALUE_DESC","status":"LISTED","steamAppName":"CSGO"}',
    'extensions': '{"persistedQuery":{"version":1,"sha256Hash":"87239fc5fa143cf0437964a20937aa558145cc8385eae48ca8734cb16abfd266"}}',
}

r = requests.get(url, headers=headers, params=params)
#print(r.text)

data = r.json()
trades = data['data']['trades']['edges']

for trade in trades:
    #print(item)
    for item in trade['node']['tradeItems']:
        print(item['marketName'])

結果：

★ Sport Gloves | Pandora's Box (Factory New)
★ StatTrak™ M9 Bayonet | Crimson Web (Factory New)
★ StatTrak™ Karambit | Crimson Web (Factory New)
★ Butterfly Knife | Sapphire (Factory New)
★ Butterfly Knife | Sapphire (Factory New)
Sticker | HellRaisers (Holo) | Katowice 2014
AWP | Dragon Lore (Factory New)
★ StatTrak™ Karambit | Emerald (Factory New)
★ Butterfly Knife | Ruby (Factory New)
★ M9 Bayonet | Emerald (Minimal Wear)
StatTrak™ M4A4 | Howl (Field-Tested)
★ Paracord Knife | Crimson Web (Factory New)
★ Paracord Knife | Crimson Web (Factory New)
★ Moto Gloves | Smoke Out (Factory New)
M4A1-S | Welcome to the Jungle (Factory New)
AWP | Dragon Lore (Battle-Scarred)
AWP | Desert Hydra (Factory New)
★ Sport Gloves | Slingshot (Minimal Wear)
★ Sport Gloves | Slingshot (Minimal Wear)
★ Classic Knife | Crimson Web (Factory New)
Sticker | compLexity Gaming (Holo) | Katowice 2014
AK-47 | Wild Lotus (Battle-Scarred)
AK-47 | Wild Lotus (Battle-Scarred)
StatTrak™ P90 | Emerald Dragon (Factory New)
StatTrak™ P90 | Emerald Dragon (Factory New)
★ StatTrak™ Skeleton Knife | Case Hardened (Minimal Wear)
★ Hand Wraps | Constrictor (Factory New)
Sticker | ESL Wolf (Foil) | Katowice 2014
Sticker | ESL Skull (Foil) | Katowice 2014
★ StatTrak™ Gut Knife | Ultraviolet (Battle-Scarred)
MP9 | Dark Age (Minimal Wear)
Souvenir P90 | Fallout Warning (Factory New)
Souvenir P250 | Gunsmoke (Factory New)
Sticker | Aerial (Gold) | Katowice 2019
StatTrak™ M4A4 | In Living Color (Battle-Scarred)
StatTrak™ M4A4 | In Living Color (Battle-Scarred)
StatTrak™ M4A4 | In Living Color (Battle-Scarred)
Sticker | TYLOO (Gold) | 2020 RMR
Sticker | yuurih (Gold) | Berlin 2019
Sticker | TYLOO (Gold) | 2020 RMR
Sticker | Natus Vincere (Gold) | 2020 RMR
Sticker | TYLOO (Gold) | 2020 RMR
Sticker | TYLOO (Gold) | 2020 RMR
M4A4 | Neo-Noir (Field-Tested)
M4A4 | Neo-Noir (Field-Tested)
StatTrak™ AWP | Atheris (Field-Tested)
CZ75-Auto | Tuxedo (Minimal Wear)
Sealed Graffiti | G2 Esports | Atlanta 2017
StatTrak™ Desert Eagle | Bronze Deco (Minimal Wear)
Sealed Graffiti | Astralis | Atlanta 2017