如何從帶有 Z80791B3AE7002CB88C2468 響應的 curl 命令的 json output 命令獲取 json output？

Question

我在 bash 腳本中使用 curl 來獲取 HTTP 標頭響應：

curl -I https://github.com
HTTP/1.1 200 OK
server: GitHub.com
date: Sun, 07 Jun 2020 06:24:11 GMT
content-type: text/html; charset=utf-8
status: 200 OK
vary: X-PJAX, Accept-Encoding, Accept, X-Requested-With
etag: W/"97a0f898499532092ef829552032f2df"
cache-control: max-age=0, private, must-revalidate
strict-transport-security: max-age=31536000; includeSubdomains; preload
x-frame-options: deny
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
referrer-policy: origin-when-cross-origin, strict-origin-when-cross-origin
expect-ct: max-age=2592000, report-uri="https://api.github.com/_private/browser/errors"

我想以 json 格式獲得上述響應。

我嘗試在 curl 命令中添加 Content-Type header：

curl -I -header "Content-Type: application/json" https://github.com

但我看不到 json 格式的 header 響應。

請幫忙。

Answer 1

curl -h | grep -e --header
 -H, --header <header/@file> Pass custom header(s) to server

所以--header不會讓響應返回為"Content-Type: application/json" 。
使用正確的 JSON 工具，您必須自己動手。 我建議你試試xidel 。

現在它真的取決於你所說的“json 格式”是什么意思。 如果您想要的只是標題列表是 JSON 數組中的單個字符串，那么可以這樣做：

xidel -s --method=HEAD https://github.com -e '[$headers[.]]'
[
  "HTTP/1.1 200 OK",
  "date: Sun, 07 Jun 2020 11:21:44 GMT",
  "content-type: text/html; charset=utf-8",
  "server: GitHub.com",
  "status: 200 OK",
  "vary: X-PJAX, Accept-Encoding, Accept, X-Requested-With, Accept-Encoding",
  "etag: W/\"3203dba51516d33a00d5d91f2099e459\"",
  "cache-control: max-age=0, private, must-revalidate",
  "strict-transport-security: max-age=31536000; includeSubdomains; preload",
  "x-frame-options: deny",
  "x-content-type-options: nosniff",
  "x-xss-protection: 1; mode=block",
  "referrer-policy: origin-when-cross-origin, strict-origin-when-cross-origin",
  "expect-ct: max-age=2592000, report-uri=\"https://api.github.com/_private/browser/errors\"",
  "content-security-policy: default-src 'none'; base-uri 'self'; block-all-mixed-content; [...]",
  "Set-Cookie: _gh_sess=OUqOJ23AXp3whSLo6MGWFYWHcUJkDOb3Z%2FgFyNySPct64QHrSYMDOJt%2F5%2BqzWeMnQ[...]",
  "Set-Cookie: _octo=GH1.1.662793087.1591528910; Path=/; Domain=github.com; Expires=Mon, 07 Jun 2021 11:21:50 GMT; Secure; SameSite=Lax",
  "Set-Cookie: logged_in=no; Path=/; Domain=github.com; Expires=Mon, 07 Jun 2021 11:21:50 GMT; HttpOnly; Secure; SameSite=Lax",
  "Accept-Ranges: bytes",
  "X-GitHub-Request-Id: 0582:2066C:2A2D3BC:3C81FF7:5EDCCDCE"
]

[.]是刪除空行（或嚴格來說是項目）。 沒有它，數組將像這樣結束：

  "Accept-Ranges: bytes",
  "X-GitHub-Request-Id: 0582:2066C:2A2D3BC:3C81FF7:5EDCCDCE"
  ""
]

如果您仍然想使用curl ，那么您可以 pipe 其 output 到xidel ：

curl -s -I https://github.com | xidel - -se '[x:lines($raw)[.]]'

x:lines($raw)是tokenize($raw,'\r\n?|\n')的簡寫，並將原始輸入$raw轉換為每個新行都是另一個項目的序列。

注意：從xidel-0.9.9-7152開始需要-e 'array{$headers[.]}'或--json-mode=deprecated <url> -e '[$headers[.]]' 。

如果您想要一個 JSON object 代替，這是可能的，但需要一些額外的步驟。
第一個字符串"HTTP/1.1 200 OK"沒有鍵名，因此您必須添加一個。 您必須將 3 個“Set-Cookie: [...]”字符串減少到 1 個，因為在 JSON object 中不允許重復鍵。

第一個帶有 JSON 鍵名“response”的字符串：

xidel -s --method=HEAD https://github.com -e '{"response":$headers[1]}'
{
  "response": "HTTP/1.1 200 OK"
}

3 個“Set-Cookie: [...]”字符串減少到 1 個：

xidel -s --method=HEAD https://github.com -e '$headers[starts-with(.,"Set-Cookie")]'
Set-Cookie: _gh_sess=OUqOJ23AXp3whSLo6MGWFYWHcUJkDOb3Z%2FgFyNySPct64QHrSYMDOJt%2F5%2BqzWeMnQ[...]; Path=/; HttpOnly; Secure; SameSite=Lax
Set-Cookie: _octo=GH1.1.662793087.1591528910; Path=/; Domain=github.com; Expires=Mon, 07 Jun 2021 11:21:50 GMT; Secure; SameSite=Lax
Set-Cookie: logged_in=no; Path=/; Domain=github.com; Expires=Mon, 07 Jun 2021 11:21:50 GMT; HttpOnly; Secure; SameSite=Lax

xidel -s --method=HEAD https://github.com -e '{"Set-Cookie":join($headers[starts-with(.,"Set-Cookie")] ! substring-after(.,"Set-Cookie: "),"; ")}'
{
  "Set-Cookie": "_gh_sess=OUqOJ23AXp3whSLo6MGWFYWHcUJkDOb3Z%2FgFyNySPct64QHrSYMDOJt%2F5%2BqzWeMnQ[...]; Path=/; HttpOnly; Secure; SameSite=Lax; _octo=GH1.1.662793087.1591528910; Path=/; Domain=github.com; Expires=Mon, 07 Jun 2021 11:21:50 GMT; Secure; SameSite=Lax; logged_in=no; Path=/; Domain=github.com; Expires=Mon, 07 Jun 2021 11:21:50 GMT; HttpOnly; Secure; SameSite=Lax"
}

換句話說： Select 所有以“Set-Cookie”開頭的標題，跳過每個項目的“Set-Cookie：” ，將它們字符串連接在一起，用a分隔; 最后創建一個鍵名為“Set-Cookie”的 JSON object 。

最后的查詢，結合其他標題並美化：

xidel -s --method=HEAD https://github.com -e '
  {|
    {
      "response":$headers[1]
    },
    for $x in $headers[.][position() gt 1][not(starts-with(.,"Set-Cookie"))] return {
      substring-before($x,":"):substring-after($x,": ")
    },
    {
      "Set-Cookie":join(
        $headers[starts-with(.,"Set-Cookie")] ! substring-after(.,"Set-Cookie: "),
        "; "
      )
    }
  |}
'
{
  "response": "HTTP/1.1 200 OK",
  "date": "Sun, 07 Jun 2020 11:21:44 GMT",
  "content-type": "text/html; charset=utf-8",
  "server": "GitHub.com",
  "status": "200 OK",
  "vary": "X-PJAX, Accept-Encoding, Accept, X-Requested-With, Accept-Encoding",
  "etag": "W/\"3203dba51516d33a00d5d91f2099e459\"",
  "cache-control": "max-age=0, private, must-revalidate",
  "strict-transport-security": "max-age=31536000; includeSubdomains; preload",
  "x-frame-options": "deny",
  "x-content-type-options": "nosniff",
  "x-xss-protection": "1; mode=block",
  "referrer-policy": "origin-when-cross-origin, strict-origin-when-cross-origin",
  "expect-ct": "max-age=2592000, report-uri=\"https://api.github.com/_private/browser/errors\"",
  "content-security-policy": "default-src 'none'; base-uri 'self'; block-all-mixed-content; [...]",
  "Accept-Ranges": "bytes",
  "X-GitHub-Request-Id": "0582:2066C:2A2D3BC:3C81FF7:5EDCCDCE",
  "Set-Cookie": "_gh_sess=OUqOJ23AXp3whSLo6MGWFYWHcUJkDOb3Z%2FgFyNySPct64QHrSYMDOJt%2F5%2BqzWeMnQ[...]; Path=/; HttpOnly; Secure; SameSite=Lax; _octo=GH1.1.662793087.1591528910; Path=/; Domain=github.com; Expires=Mon, 07 Jun 2021 11:21:50 GMT; Secure; SameSite=Lax; logged_in=no; Path=/; Domain=github.com; Expires=Mon, 07 Jun 2021 11:21:50 GMT; HttpOnly; Secure; SameSite=Lax"
}

- {| |}將多個鍵值對組合在一起。
- 文字中的其他標題（中間部分）：對於不以“Set-Cookie”開頭的所有（非空）標題（第一個除外），創建一個 JSON 鍵值對，鍵為字符串在:之前和它后面的值。