使用 python 將 JSON 發送到帶有時間戳字段的 ElasticSearch 索引

Question

我有一個腳本可以調用我的 API 並提取股票數據。 我想將來自 API 的響應保存為 JSON 文件，然后使用響應中的“t”字段作為日期字段/時間戳創建 ES 索引。

我可以看到 ES 集群中的數據，但它沒有被索引，並且“t”字段顯示為錯誤的類型。 長而不是日期。

我不確定如何最好地索引results因為這對我來說很關鍵，因為我真的想將其顯示為時間序列。

收藏家.py

def collect():
    import json
    key = ""
    url = "https://api"
    payload={}
    headers = {}
    response = requests.request("GET", url, headers=headers, data=payload)
    j = response.json()
    print (j['results'][1])
    data = response.text
    for i in j['results']: 
        print (i)
    with open('data.json', 'w', encoding='utf-8') as f:
        json.dump(j, f, ensure_ascii=False, indent=4)
    import os, sys
    from elasticsearch import Elasticsearch
    directory = '/home/'
    res = requests.get('http://localhost:9200')    
    print (res.content)
    es = Elasticsearch([{'host': 'localhost', 'port': '9200'}])
    i = 1
    for filename in os.listdir(directory):
        if filename.endswith(".json"):
         f = open(filename)
         docket_content = f.read()
         # Send the data into es
         es.index(index='core', ignore=400, doc_type='docket', 
         id=i, body=json.loads(docket_content))
         i = i + 1
    return render_template('show.html', data=data)

JSON 響應和 data.json 的內容

{
  "ticker": "AAPL",
  "queryCount": 10,
  "resultsCount": 10,
  "adjusted": true,
  "results": [
    {
      "v": 1668,
      "vw": 151.8826,
      "a": 151.8826,
      "o": 152,
      "c": 151.92,
      "h": 152,
      "l": 151.75,
      "t": 1636012800000,
      "n": 70
    },
    {
      "v": 1467,
      "vw": 151.9323,
      "a": 151.9059,
      "o": 151.95,
      "c": 151.96,
      "h": 151.96,
      "l": 151.89,
      "t": 1636012860000,
      "n": 60
    },
    {
      "v": 1096,
      "vw": 151.9585,
      "a": 151.9195,
      "o": 151.96,
      "c": 151.94,
      "h": 151.96,
      "l": 151.94,
      "t": 1636012920000,
      "n": 64
    },
    {
      "v": 1303,
      "vw": 151.7871,
      "a": 151.8889,
      "o": 151.77,
      "c": 151.73,
      "h": 151.77,
      "l": 151.73,
      "t": 1636013040000,
      "n": 70
    },
    {
      "v": 847,
      "vw": 151.8279,
      "a": 151.8811,
      "o": 151.87,
      "c": 151.8,
      "h": 151.87,
      "l": 151.8,
      "t": 1636013100000,
      "n": 37
    },
    {
      "v": 451,
      "vw": 151.8722,
      "a": 151.8737,
      "o": 151.87,
      "c": 151.87,
      "h": 151.87,
      "l": 151.87,
      "t": 1636013280000,
      "n": 17
    },
    {
      "v": 5347,
      "vw": 151.8021,
      "a": 151.8446,
      "o": 151.82,
      "c": 151.8,
      "h": 151.82,
      "l": 151.8,
      "t": 1636013400000,
      "n": 55
    },
    {
      "v": 2834,
      "vw": 151.7431,
      "a": 151.8266,
      "o": 151.74,
      "c": 151.73,
      "h": 151.74,
      "l": 151.73,
      "t": 1636013460000,
      "n": 58
    },
    {
      "v": 615,
      "vw": 151.7193,
      "a": 151.8226,
      "o": 151.73,
      "c": 151.68,
      "h": 151.73,
      "l": 151.68,
      "t": 1636013520000,
      "n": 22
    },
    {
      "v": 876,
      "vw": 151.717,
      "a": 151.8173,
      "o": 151.71,
      "c": 151.73,
      "h": 151.73,
      "l": 151.71,
      "t": 1636013580000,
      "n": 33
    }
  ],
  "status": "OK",
  "request_id": "2354523452356236",
  "count": 10
}

彈性指數

core    
mappings    
properties  
a   
type    "float"
c   
type    "float"
h   
type    "float"
l   
type    "float"
n   
type    "long"
o   
type    "float"
t   
type    "long"
v   
type    "long"
vw  
type    "float"

Answer 1

你真的應該在索引之前定義你的映射，要么使用模板，要么使用集合映射發布索引。 否則 Elasticsearch 將嘗試為每個字段選擇最佳數據類型，但它並不總是正確的，如您所見

https://elasticsearch-py.readthedocs.io/en/v7.15.1/api.html?highlight=mapping#elasticsearch.client.IndicesClient.put_mapping可能就是你想要的