如何从 Elasticsearch 中删除文档

Question

I can't find any example of deleting documents from Elasticsearch in Python. What I've seen by now - is definition of delete and delete_by_query functions. But for some reason documentation does not provide even a microscopic example of using these functions. The single list of parameters does not tell me too much, if I do not know how to correctly feed them into the function call. So, lets say, I've just inserted one new doc like so:

doc = {'name':'Jacobian'}
db.index(index="reestr",doc_type="some_type",body=doc)

Who in the world knows how can I now delete this document using delete and delete_by_query ?

Answer 1

Since you are not giving a document id while indexing your document, you have to get the auto-generated document id from the return value and delete according to the id. Or you can define the id yourself, try the following:

 db.index(index="reestr",doc_type="some_type",id=1919, body=doc)

 db.delete(index="reestr",doc_type="some_type",id=1919)

In the other case, you need to look into return value;

 r = db.index(index="reestr",doc_type="some_type", body=doc)
 # r = {u'_type': u'some_type', u'_id': u'AU36zuFq-fzpr_HkJSkT', u'created': True, u'_version': 1, u'_index': u'reestr'}

 db.delete(index="reestr",doc_type="some_type",id=r['_id'])

Another example for delete_by_query. Let's say after adding several documents with name='Jacobian', run the following to delete all documents with name='Jacobian':

 db.delete_by_query(index='reestr',doc_type='some_type', q={'name': 'Jacobian'})

Answer 2

The Delete-By-Query API was removed from the ES core in version 2 for several reasons. This function became a plugin. You can look for more details here:

Why Delete-By-Query is a plugin

Delete By Query Plugin

Because I didn't want to add another dependency (because I need this later to run in a docker image) I wrote an own function solving this problem. My solution is to search for all quotes with the specified index and type. After that I remove them using the Bulk API:

def delete_es_type(es, index, type_):
    try:
        count = es.count(index, type_)['count']
        response = es.search(
            index=index,
            filter_path=["hits.hits._id"],
            body={"size": count, "query": {"filtered" : {"filter" : {
                  "type" : {"value": type_ }}}}})
        ids = [x["_id"] for x in response["hits"]["hits"]]
        if len(ids) > 0:
            return
        bulk_body = [
            '{{"delete": {{"_index": "{}", "_type": "{}", "_id": "{}"}}}}'
            .format(index, type_, x) for x in ids]
        es.bulk('\n'.join(bulk_body))
        # es.indices.flush_synced([index])
    except elasticsearch.exceptions.TransportError as ex:
        print("Elasticsearch error: " + ex.error)
        raise ex

I hope that helps future googlers ;)

Answer 3

One can also do something like this:

def delete_by_ids(index, ids):
    query = {"query": {"terms": {"_id": ids}}}
    res = es.delete_by_query(index=index, body=query)
    pprint(res)

# Pass index and list of id that you want to delete.
delete_by_ids('my_index', ['test1', 'test2', 'test3'])

Which will perform the delete operation on bulk data

Answer 4

I came across this post while searching for a way to delete a document on ElasticSearch using their Python library, ElasticSearch-DSL.

In case it helps anyone, this part of their documentation describes the document lifecycle. https://elasticsearch-dsl.readthedocs.io/en/latest/persistence.html#document-life-cycle

And at the end of it, it details how to delete a document:

To delete a document just call its delete method:

 first = Post.get(id=42) first.delete()

Hope that helps 🤞