如何使用 python 客户端获取 elasticsearch 索引下的所有文档？

Question

I'm trying to get all index document using python client but the result show me only the first document This is my python code:

res = es.search(index="92c603b3-8173-4d7a-9aca-f8c115ff5a18", doc_type="doc", body = {
'size' : 10000,
'query': {
    'match_all' : {}
}
})
print("%d documents found" % res['hits']['total'])
data = [doc for doc in res['hits']['hits']]
for doc in data:
    print(doc)
    return "%s %s %s" % (doc['_id'], doc['_source']['0'], doc['_source']['5'])

Answer 1

Elasticsearch by default retrieve only 10 documents. You could change this behaviour - doc here . The best practice for pagination are search after query and scroll query . It depends from your needs. Please read this answer Elastic search not giving data with big number for page size

To show all the results:

for doc in res['hits']['hits']:
    print doc['_id'], doc['_source']

Answer 2

try "_doc" instead of "doc"

res = es.search(index="92c603b3-8173-4d7a-9aca-f8c115ff5a18", doc_type="_doc", body = {
'size' : 100,
'query': {
    'match_all' : {}
}
})

Answer 3

You can try the following query. It will return all the documents.

result = es.search(index="index_name", body={"query":{"match_all":{}}})

Answer 4

You can also use elasticsearch_dsl and its Search API which allows you to iterate over all your documents via the scan method.

import elasticsearch
from elasticsearch_dsl import Search

client = elasticsearch.Elasticsearch()
search = Search(using=client, index="92c603b3-8173-4d7a-9aca-f8c115ff5a18")

for hit in search.scan():
    print(hit)

Answer 5

I dont see mentioned that the index must be refreshed if you just added data. Use this:

es.indices.refresh(index="index_name")