Using Logstash to delete an index from ES

Question

I want to delete an index from ES before uploading a updated version of the data from MySQL . As of today I use curl to delete the data and Logstash to upload, however I was wondering if there is a way where I do not have to do it manually separated, best case being able to use Logstash (with its scheduler) to have it fully automatic. Say is there a way to simply write something in my config-file to do this?

How I´m deleting today :

curl -X -DELETE 'http://localhost:9200/index'

The reason to why I am deleting the index before uploading new data is because I am uploading data that is coming from various sources, hence the ability to decide when the data was updated is not that easy. This is why I have chosen to delete the data before uploading a updated version. If there is a smarter way to approach this problem I would love to hear it.

Answer 1

You can create a batch file that runs the logstash process and curator actions together.

Download and install curator using the instructions. Use curator command line to perform actions on indices. For deletion, check out Delete Indices action.

action: delete_indices
description: >-
  Delete indices older than 45 days (based on index name), for logstash-
  prefixed indices. Ignore the error if the filter does not result in an
  actionable list of indices (ignore_empty_list) and exit cleanly.
options:
  ignore_empty_list: True
  disable_action: True
filters:
- filtertype: pattern
  kind: prefix
  value: %{Index_Name}
- filtertype: age
  source: name
  direction: older
  timestring: '%Y.%m.%d'
  unit: days
  unit_count: 45

Source - Delete Indices Example