堆棧內存溢出
  簡體   English   中英

Python 請求模擬 CURL POST 發送帶有 1 個或多個文件和 JSON 正文的多部分請求

[英]Python Requests Emulate a CURL POST sending multipart request with 1 or more files AND JSON body

 原文  2019-02-12 18:42:24  0   1    java/ python-2.7/ spring-boot/ curl/ python-2.x

問題描述

我已經在這方面進行了兩天的黑客攻擊，但沒有運氣！

工作卷曲請求

curl -X POST -v "http://$1:8080/controller/endpoint" -H "Cache-Control: no-cache" -H "Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW" -F "message={ \"id\": \"b3562c86-6ff4-4bf7-9c4a-4c64fff4d0ea\", \"stuff\": [
{
\"id\": \"1ca2d9b1-1d73-432a-b483-be404afff8da\",
.......
\"endTime\": \"\"
}]}};type=application/json" -F "files=@file.zip"

返回如下所示的輸出：

 ./rest.sh http://127.0.0.1/anything
* Hostname was NOT found in DNS cache
*   Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
> POST /anything HTTP/1.1
> User-Agent: curl/7.35.0
> Host: 127.0.0.1
> Accept: */*
> Cache-Control: no-cache
> Content-Length: 493
> Expect: 100-continue
> Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW; boundary=------------------------52912a6946761b42
>
< HTTP/1.1 100 Continue
< HTTP/1.1 200 OK
* Server gunicorn/19.9.0 is not blacklisted
< Server: gunicorn/19.9.0
< Date: Tue, 12 Feb 2019 18:18:56 GMT
< Connection: keep-alive
< Content-Type: application/json
< Content-Length: 725
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Credentials: true
<
{
  "args": {},
  "data": "",
  "files": {
    "files": "ZIP-CONTENT-GOES-HERE"
  },
  "form": {
    "message": "{ \"runId\": \"1ca2d9b1-1d73-432a-b483-be404a13e8da\", \"reports\": [\n{\n\"executionId\": \"1ca2d9b1-1d73-432a-b483-be404a13e8da\",\n\"endTime\": \"\"\n}]}}"
  },
  "headers": {
    "Accept": "*/*",
    "Cache-Control": "no-cache",
    "Content-Length": "493",
    "Content-Type": "multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW; boundary=------------------------52912a6946761b42",
    "Expect": "100-continue",
    "Host": "127.0.0.1",
    "User-Agent": "curl/7.35.0"
  },
  "json": null,
  "method": "POST",
  "origin": "172.17.42.1",
  "url": "http://127.0.0.1/anything"
}
* Connection #0 to host 127.0.0.1 left intact

現在，如果我將,scrub2.zip添加到 curl 命令（發送 2 個 zip 文件和 JSON 數據），我會得到如下所示的輸出：

 ./rest.sh http://127.0.0.1/anything
* Hostname was NOT found in DNS cache
*   Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
> POST /anything HTTP/1.1
> User-Agent: curl/7.35.0
> Host: 127.0.0.1
> Accept: */*
> Cache-Control: no-cache
> Content-Length: 878
> Expect: 100-continue
> Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW; boundary=------------------------27d684afce904423
>
< HTTP/1.1 100 Continue
< HTTP/1.1 200 OK
* Server gunicorn/19.9.0 is not blacklisted
< Server: gunicorn/19.9.0
< Date: Tue, 12 Feb 2019 18:20:36 GMT
< Connection: keep-alive
< Content-Type: application/json
< Content-Length: 1117
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Credentials: true
<
{
  "args": {},
  "data": "",
  "files": {},
  "form": {
    "files": "--------------------------fd702594c1765b85\r\nContent-Disposition: attachment; filename=\"scrubbed.zip\"\r\nContent-Type: application/octet-stream\r\n\r\nZIP-CONTENT-GOES-HERE\r\n--------------------------fd702594c1765b85\r\nContent-Disposition: attachment; filename=\"scrubbed2.zip\"\r\nContent-Type: application/octet-stream\r\n\r\nZIP-CONTENT-GOES-HERE222222222\n\r\n--------------------------fd702594c1765b85--",
    "message": "{ \"runId\": \"1ca2d9b1-1d73-432a-b483-be404a13e8da\", \"reports\": [\n{\n\"executionId\": \"1ca2d9b1-1d73-432a-b483-be404a13e8da\",\n\"endTime\": \"\"\n}]}}"
  },
  "headers": {
    "Accept": "*/*",
    "Cache-Control": "no-cache",
    "Content-Length": "878",
    "Content-Type": "multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW; boundary=------------------------27d684afce904423",
    "Expect": "100-continue",
    "Host": "127.0.0.1",
    "User-Agent": "curl/7.35.0"
  },
  "json": null,
  "method": "POST",
  "origin": "172.17.42.1",
  "url": "http://127.0.0.1/anything"
}
* Connection #0 to host 127.0.0.1 left intact

你看得到差別嗎？ 這兩個文件現在嵌入在表單/文件中，而不是單獨顯示文件和表單/消息！

此類 CURL 請求在 Java API 端點上被接受，在調試器中如下所示： CURL 發送時兩個文件在 Java 調試器中的顯示方式

但我對 Python 的所有嘗試，例如：

multipart_form_data_object = {
    'scrubbed.zip': (args.files[0], open(args.files[0], 'rb'), "application/json"),
    'files': (args.files[1], open(args.files[1], 'rb'), "application/json"),
    'message': (None, open(args.message, 'rb'), 'application/json')
}
 response = requests.post(args.url + ':' + str(args.port) + '/' + args.endpoint, files=multipart_form_data_object,
                             proxies=proxies)

（這是我讓它最接近的工作），看起來像這樣：

multipart_form_data_object = {
    'scrubbed.zip': (args.files[0], open(args.files[0], 'rb'), "application/json"),
    'files': (args.files[1], open(args.files[1], 'rb'), "application/json"),
    'message': (None, open(args.message, 'rb'), 'application/json')
}
response = requests.post(args.url + ':' + str(args.port) + '/' + args.endpoint, files=multipart_form_data_object,
                         proxies=proxies)

輸出為：

{'Content-Length': '664', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'User-Agent': 'python-requests/2.21.0', 'Connection': 'keep-alive', 'Content-Type': 'multipart/form-data; boundary=227d4ef5a41db8a690e5cebadf336851'}
{
  "args": {},
  "data": "",
  "files": {
    "files": "ZIP-CONTENT-GOES-HERE",
    "scrubbed.zip": "ZIP-CONTENT-GOES-HERE22222"
  },
  "form": {
    "message": "{\r\n  \"runId\": \"9c4a-4c64f6d4d0ea\",\r\n  \"reports\": [\r\n    {\r\n      \"executionId\": \"d73-432a-b483-be404a13e8da\",\r\n      \"endTime\": \"\"\r\n    }\r\n  ]\r\n}"
  },
  "headers": {
    "Accept": "*/*",
    "Accept-Encoding": "gzip, deflate",
    "Connection": "keep-alive",
    "Content-Length": "664",
    "Content-Type": "multipart/form-data; boundary=227d4ef5a41db8a690e5cebadf336851",
    "Host": "java.api.host.com",
    "User-Agent": "python-requests/2.21.0"
  },
  "json": null,
  "method": "POST",
  "origin": "10.0.0.2",
  "url": "http://java.api.host.com/anything"
}

現在，嘗試調整它以發送一組文件（否則，如果我將scrubbed.zip重命名為files ，它會被覆蓋），使其看起來像：

multipart_form_data_object = {
    'files': [(args.files[0], open(args.files[0], 'rb'), "application/json"),
     (args.files[1], open(args.files[1], 'rb'), "application/json")],
    'message': (None, open(args.message, 'rb'), 'application/json')
}

導致錯誤：

Traceback (most recent call last):
  File ".\load_stress_test_endpoint.py", line 84, in <module>
    post()
  File ".\load_stress_test_endpoint.py", line 76, in post
    proxies=proxies)
  File "C:\Python\Python27\lib\site-packages\requests\api.py", line 116, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "C:\Python\Python27\lib\site-packages\requests\api.py", line 60, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Python\Python27\lib\site-packages\requests\sessions.py", line 519, in request
    prep = self.prepare_request(req)
  File "C:\Python\Python27\lib\site-packages\requests\sessions.py", line 462, in prepare_request
    hooks=merge_hooks(request.hooks, self.hooks),
  File "C:\Python\Python27\lib\site-packages\requests\models.py", line 316, in prepare
    self.prepare_body(data, files, json)
  File "C:\Python\Python27\lib\site-packages\requests\models.py", line 504, in prepare_body
    (body, content_type) = self._encode_files(files, data)
  File "C:\Python\Python27\lib\site-packages\requests\models.py", line 169, in _encode_files
    body, content_type = encode_multipart_formdata(new_fields)
  File "C:\Python\Python27\lib\site-packages\urllib3\filepost.py", line 90, in encode_multipart_formdata
    body.write(data)
TypeError: 'tuple' does not have the buffer interface

我最后的嘗試是不同的數據結構（列表），如下：

multiple_files_list = [
    ('files', (args.files[0], open(args.files[0], 'rb'), "application/json")),
    ('files', (args.files[1], open(args.files[1], 'rb'), "application/json")),
    ('message', None, open(args.message, 'rb'), 'application/json')
]

結果報錯：

Traceback (most recent call last):
  File ".\load_stress_test_endpoint.py", line 84, in <module>
    post()
  File ".\load_stress_test_endpoint.py", line 76, in post
    proxies=proxies)
  File "C:\Python\Python27\lib\site-packages\requests\api.py", line 116, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "C:\Python\Python27\lib\site-packages\requests\api.py", line 60, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Python\Python27\lib\site-packages\requests\sessions.py", line 519, in request
    prep = self.prepare_request(req)
  File "C:\Python\Python27\lib\site-packages\requests\sessions.py", line 462, in prepare_request
    hooks=merge_hooks(request.hooks, self.hooks),
  File "C:\Python\Python27\lib\site-packages\requests\models.py", line 316, in prepare
    self.prepare_body(data, files, json)
  File "C:\Python\Python27\lib\site-packages\requests\models.py", line 504, in prepare_body
    (body, content_type) = self._encode_files(files, data)
  File "C:\Python\Python27\lib\site-packages\requests\models.py", line 141, in _encode_files
    for (k, v) in files:
ValueError: too many values to unpack

能否請您指點一下，如何使 Python 請求包執行與 CURL 請求類似的操作？

以下是 Java 端點的設置方式：

public Response index(@RequestPart("message") @Valid
                          final Message message,
                          @ApiParam(value = "Multipart File array of compressed archives (zip) ", required = true) @RequestPart("files") @Valid
                          final MultipartFile[] files)

1 個解決方案

解決方案1
 1 已采納  2019-02-14 09:56:26

您的腳本應如下所示：

注意：有一個對requests_toolbelt的依賴

發送.py

import argparse
import requests
from requests_toolbelt import MultipartEncoder

parser = argparse.ArgumentParser()
parser.add_argument('message')
parser.add_argument('--files', nargs='+')
args = parser.parse_args()

multipart_form_data_object = MultipartEncoder(
    fields=(
        ('files', (args.files[0], open(args.files[0], 'rb'), "application/json")),
        ('files', (args.files[1], open(args.files[1], 'rb'), "application/json")),
        ('message', ('message', open(args.message, 'rb'), 'application/json')),
    )
)

res = requests.post('http://localhost:8000', data=multipart_form_data_object, headers={'Content-Type': multipart_form_data_object.content_type})
print(res.content)

我使用 django 測試了它：

網址.py

from django.urls import path
from django.http import JsonResponse
from django.views.decorators.csrf import csrf_exempt

@csrf_exempt
def dump(request):
    data = {name: [o.read().decode('utf8') for o in request.FILES.getlist(name)] for name in request.FILES.keys()}
    return JsonResponse(data)

urlpatterns = [
    path('', dump),
]

調用它使用：

curl -s http://127.0.0.1:8000/ -F "message=@$(pwd)/file1" -F "files=@$(pwd)/file2" -F "files=@$(pwd)/file3"

並使用 python

python send.py file1 --files file2 file3

相同的輸出：

{"files": ["{\\"message\\": \\"hello world\\"}\\n", "something else\\n"], "message": ["hello world\\n"]}

暫無
暫無

聲明:本站的技術帖子網頁，遵循CC BY-SA 4.0協議，如果您需要轉載，請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.

相關問題 使用 json 正文和多部分文件數組的 PUT 請求 分段JSON POST請求 Postman 與“正文”（json）和文件（多部分）一起在 1 個 restful 請求中 從 Java 中的 Curl Post 請求中獲取 Json / Resonse 正文 Android java 發送 JSON 發布請求，正文為空 使用JSON正文發送POST請求時出現400響應 通過curl命令發送POST請求 使用帶有JSON正文的Restlet進行多部分POST處理 NET中發送帶有身體的POST請求 無法發布多部分表單數據json請求
相關標簽
 
粵ICP備18138465號  © 2020-2024 STACKOOM.COM