Getting error in logstash while pushing data into Elasticsearch using amazon_es plugin

Question

I am trying to push data into Amazon Elasticsearch using Logstash. But getting the error -

Error while performing resurrection {:error_message=>"undefined method `credentials' for nil:NilClass", :class=>"NoMethodError", :backtrace=>["/etc/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.32/lib/aws-sdk-core/signers/v4.rb:45:in `initialize'", "/etc/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/http_client/manticore_adapter.rb:108:in `perform_request'", "/etc/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/http_client/pool.rb:291:in `perform_request_to_url'", "/etc/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/http_client/pool.rb:245:in `block in healthcheck!'", "org/jruby/RubyHash.java:1343:in `each'", "/etc/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/http_client/pool.rb:241:in `healthcheck!'", "/etc/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.1.1-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:234:in `block in start_resurrectionist'", "/etc/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.1.1-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:128:in `until_stopped'", "/etc/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.1.1-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:233:in `block in start_resurrectionist'"]}

Logstash version - 6.2.4 Elasticsearch version - 6.5

Config is given below -

amazon_es
  {
               id => 'es-bag-prod'
               hosts => ["https://${ES_HOST}:${ES_PORT}"]
               region => "ap-southeast-1"
               aws_access_key_id => "${AWS_ACCESS_KEY}"
               aws_secret_access_key => "${AWS_SECRET_KEY}"
               timeout => 25
               index => "${ES_INDEX_PREFIX_BAG}-%{[@metadata][month]}"
               document_type => "${ES_BAG_DOC}"
               document_id => "%{[@metadata][doc_id]}"
               action => "%{[@metadata][action]}"
               version => "%{[@metadata][version]}"
               version_type => "external_gte"
 }

Answer 1

How are you starting Logstash? It seems that there maybe issues with access to the env variables when running Logstash as a service. See this for more info:

https://github.com/awslabs/logstash-output-amazon_es/issues/34