使用python从xhr请求中获取数据

Question

我试图在网站https://www.jiocinema.com/search/avengers上获取电影和连续剧 我已经使用 selenium 提取了电影，但我了解了 xhr 请求。 我是这个概念的新手，不知道我是否可以使用 api ？

API链接为： https : //prod.media.jio.com/apis/common/v3.1/search/search

xhr 响应看起来像

有什么办法可以从上述 xhr 响应中获取数据？

相关： Python，从网站提取 XHR 响应数据

Answer 1

您可以使用请求库来发出这样的帖子请求......

import requests

headers = {'User-Agent':'Some user agent'}
data = requests.post('https://prod.media.jio.com/apis/common/v3.1/search/search',headers=headers).text

您可能需要标头来发出请求...

Answer 2

你实际上不需要硒。 您在此处调用REST -API。

简单地做这样的事情：

import requests
import traceback

def searchApi(query):
    endpoint = "http://prod.media.jio.com/apis/common/v3.1/search/auto"
    data = {
        "q": query
    }
    try:
        response = requests.post(endpoint, data=data)
        if(response.status_code == 200):
            for msg in response:
                print(msg)
    except Exception:
        print(traceback.format_exc())

用法：

searchApi("avengers")

原始输出：

{
    "code": 200,
    "message": "success",
    "data": {
        "items": [
            {
                "name": "avengers grimm",
                "type": "Movies"
            },
            {
                "name":"avengers  endgame   official trailer  hindi ",
                "type":"Videos"
            },
            {
                "name":"avengers  endgame   official trailer",
                "type":"Videos"
            },
            {
                "name":"avengers endgame   special look",
                "type":"Videos"
            }
            .... continues
        ]
    }
}

或者，如果您想直接访问数据响应。

import json

def searchApi(query):
    endpoint = "http://prod.media.jio.com/apis/common/v3.1/search/auto"
    data = {
        "q": query
    }
    try:
        response = requests.post(endpoint, data=data)
        if(response.status_code == 200):
            response = response.json()
            for msg in response["data"]["items"]:
                print("name: ", msg["name"], "type: ", msg["type"])
    except Exception:
        print(traceback.format_exc())

格式化输出msg["name"]和msg["type"] ：

name:  avengers grimm type:  Movies
name:  avengers  endgame   official trailer type:  Videos
name:  avengers endgame   special look type:  Videos
name:  avengers  endgame   official trailer  hindi  type:  Videos
name:  the avengers  earth s mightiest heroes type:  TV Shows
name:  marvel's avengers  age of ultron type:  Movies
name:  marvel's avengers assemble type:  TV Shows
name:  marvel's avengers  age of ultron   official trailer  hindi  type:  Videos
name:  marvel's avengers  age of ultron   official trailer type:  Videos
name:  marvel's the avengers type:  Movies
name:  marvel's the avengers   official trailer type:  Videos
name:  marvel's the avengers official trailer   hindi type:  Videos
name:  making of south indian avengers type:  Videos