Python請求與curl

Question

我正在嘗試使用具有特定標頭和正文有效負載的POST從URL中提取數據。 使用curl或Advanced Rest Client之類的工具，我可以獲取包含所需數據和漂亮代碼200的JSON。但是，如果我嘗試使用python請求執行相同的請求，它將無法正常工作。 我收到代碼405和錯誤的json。

這些是我正在使用的配置：

curl -H "Content-Type: application/json" --data @payload.json -X POST https://www.walmart.com/terra-firma/fetch?rgs=REVIEWS_MAP

json文件payload.json如下所示：

{"productId":"10VJLBWPW82T","paginationContext":{"sort":"relevancy","page":3}}

這是python3代碼：

import requests

url = "http://www.walmart.com/terra-firma/fetch?rgs=REVIEWS_MAP"

head = {'Content-type':'application/json'}
payload = {"productId":"10VJLBWPW82T","paginationContext":{"sort":"relevancy","page":3}}

rep = requests.post(url, headers=head, data=payload)

有人知道如何解釋這種差異嗎？

Answer 1

有兩件事，首先在請求中使用'json'參數，而不是用於發布json的'data'參數。 其次，對該特定網頁使用https而非http協議。

import requests

url = "https://www.walmart.com/terra-firma/fetch?rgs=REVIEWS_MAP"
head = {'Content-type':'application/json'}
payload = {"productId":"10VJLBWPW82T","paginationContext":{"sort":"relevancy","page":3}}
rep = requests.post(url, headers=head, json=payload)
print (rep.status_code)
print (rep.json())

輸出：

200
{'status': 'OK', 'header': {'headerAttributes': {'timing': {'total': {'threads': 3, 'orchestrate': 1, 'delay': 0, 'elapsed': 30, 'futures': 37}}, 'tenant': {'WM_BU_ID': '0', 'WM_VERTICAL_ID': '0', 'WM_LOCALE_ID': 'en_US', 'WM_MART_ID': '0', 'WM_CHANNEL_ID': '0'}, 'server': {'ver': '84.0.2', 'env': 'prod-a', 'dc': 'prod-dfw8', 'name': 'app-273848643-9-280204056', 'sha': 'bd15b234fd7d120a2aa489becec41385c5e34719'}, 'context': {'browser': 'CRAWLER', 'device': 'UNSPECIFIED', 'topTxId': 'b0886fbe-1e8-161e8486db3000', 'access': True}}}, 'errors': [], 'payload': {'products': {}, 'reviews': {'10VJLBWPW82T': {'percentageFourCount': 2, 'percentageFiveCount': 91, 'ratingValueTwoCount': 1, 'customerReviews': [{'showRecommended': True, 'photos': [], 'userNickname': 'murrie56', 'positiveFeedback': 5, 'negativeFeedback': 0, 'userAttributes': {}, 'rating': 5.0, 'authorId': '6b61e60fc8b3f358b84858dbe0fd50f7', 'reviewTitle': 'LOVE IT! LOVE IT! LOVE IT!', 'reviewText': "I've been using this ever since it was introduced back in the late 70's, I was in High School. I still get compliments on it. I'll never wear anything else as long as it's available. I buy in bulk.", 'videos': [], 'reviewSubmissionTime': '11/30/2014', 'reviewId': '40087713', 'recommended': True}, {'showRecommended': False, 'photos': [], 'userNickname': 'Tony', 'positiveFeedback': 0, 'negativeFeedback': 0, 'userAttributes': {}, 'rating': 5.0, 'authorId': 'ce2dbcdd5602bd2fde14f2d40bfc3217', 'reviewText': 'my girl friend loved it', 'videos': [], 'reviewSubmissionTime': '12/18/2017', 'reviewId': '186608534'}, {'showRecommended': True, 'photos': [], 'userNickname': 'SandyC', 'positiveFeedback': 5, 'badges': [{'id': 'VerifiedPurchaser', 'badgeType': 'Custom', 'contentType': 'REVIEW'}], 'userAge': '45 - 54', 'negativeFeedback': 0, 'userAttributes': {'Usage': 'Every day', 'Age': '45 - 54', 'Ownership': 'Longer than one year', 'Gender': 'Female'}, 'rating': 5.0, 'authorId': '6a0c52dc6fc5c63ccff7bde8ad1f3e9a', 'reviewTitle': 'Have been using this perfume for years', 'reviewText': 'Love it!', 'userLocation': 'Barnwell, SC', 'userGender': 'Female', 'reviewSubmissionTime': '12/21/2013', 'reviewId': '19488031', 'videos': [], 'recommended': True}, {'showRecommended': True, 'photos': [], 'userNickname': 'corkmiester', 'positiveFeedback': 6, 'negativeFeedback': 1, 'userAttributes': {}, 'rating': 5.0, 'authorId': '95edbfc402035dd0237b48ef92a5401f', 'reviewTitle': 'Lifetime Favorite', 'reviewText': "I've used this product since my teenage years, and I'm happy to say that the scent has never varied, never changed. It smells the same as it did then and is my all-time favorite perfume. I love the staying power of this product. An application in the morning lasts all day. I've had many a compliment over the years when wearing this and the affordable price makes it that much better. I hope they never stop making this stuff. I'll be a happy camper as long as I can find it. :)", 'videos': [], 'reviewSubmissionTime': '12/10/2014', 'reviewId': '41086799', 'recommended': True}, {'showRecommended': True, 'photos': [], 'userNickname': 'Clydesdales', 'positiveFeedback': 4, 'negativeFeedback': 0, 'userAttributes': {}, 'rating': 5.0, 'authorId': '0b1608e40c638ce49215bfba6624e27d', 'reviewTitle': 'GL McC', 'reviewText': 'Have used this cologne for years and I can always get it at Walmart. The other places I have gotten it before no longer carry the product. Thanks Walmart for carrying this item!!', 'videos': [], 'reviewSubmissionTime': '3/30/2016', 'reviewId': '84475295', 'recommended': True}], 'overallRatingRange': 5.0, 'totalReviewCount': 45, 'activeFilters': {}, 'percentageTwoCount': 2, 'roundedAverageOverallRating': 4.8, 'recommendedPercentage': 95, 'ratingValueFourCount': 1, 'pagination': {'total': 43, 'currentSpan': '11-15', 'next': {'num': 0, 'url': 'sort=relevancy&page=4', 'gap': False, 'active': False}, 'pages': [{'num': 1, 'url': 'sort=relevancy&page=1', 'gap': False, 'active': False}, {'num': 2, 'url': 'sort=relevancy&page=2', 'gap': False, 'active': False}, {'num': 3, 'url': 'sort=relevancy&page=3', 'gap': False, 'active': True}, {'num': 4, 'url': 'sort=relevancy&page=4', 'gap': False, 'active': False}, {'num': 5, 'url': 'sort=relevancy&page=5', 'gap': False, 'active': False}, {'num': 6, 'url': 'sort=relevancy&page=6', 'gap': False, 'active': False}, {'num': 0, 'gap': True, 'active': False}, {'num': 9, 'url': 'sort=relevancy&page=9', 'gap': False, 'active': False}], 'previous': {'num': 0, 'url': 'sort=relevancy&page=2', 'gap': False, 'active': False}}, 'percentageOneCount': 2, 'ratingValueFiveCount': 41, 'ratingValueThreeCount': 1, 'ratingValueOneCount': 1, 'activeSort': 'relevancy', 'percentageThreeCount': 2, 'averageOverallRating': 4.7778}}}}