logstash access with readonly rest plugin

Question

we have a problem with the readonly rest plugin for elasticsearch: we don't get logstash running when the plugin is enabled. We use logstash with filebeat. Can this be the problem? The logstash config is below. The error message:

[401] Forbidden {:class=>"Elasticsearch::Transport::Transport::Errors::Unauthorized", :level=>:error}

In elasticsearch we have defined the roles as you see below.

readonlyrest:
   enable: true
   response_if_req_forbidden: <h1>Forbidden</h1>    
   access_control_rules:
    - name: Developer (reads only logstash indices, but can create new charts/dashboards)
      auth_key: dev:dev
      type: allow
      kibana_access: ro+
      indices: ["<no-index>", ".kibana*", "logstash*", "default"]
   - name: Kibana Server (we trust this server side component, full access granted via HTTP authentication)
     auth_key: admin:passwd1
     type: allow
   - name: "Logstash can write and create its own indices"
     auth_key: logstash:logstash
     type: allow
     actions: ["cluster:*", "indices:data/read/*","indices:data/write/*","indices:admin/*"]
     indices: ["logstash*", "filebeat-*", "<no_index>"]
the logstash config:

output{
    elasticsearch {      
    hosts => ["localhost:9200"]
        manage_template => true
        index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
        document_type => "%{[@metadata][type]}"
        document_id => "%{fingerprint}"
    user => ["logstash"]
    password => ["logstash"]
    }
}

Answer 1

I believe you are not giving logstash the ability to create indexes with your setup. It can write and read, but I am not seeing create.

From the example of the website, can you change your logstash config to:

-  name: "Logstash can write and create its own indices"
   auth_key: logstash:logstash
   type: allow
   actions: ["indices:data/read/*","indices:data/write/*","indices:admin/template/*","indices:admin/create"]
   indices: ["logstash-*", "<no_index>"]

This setup works for me.

I don't think it has anything to do with filebeat since the output doesn't actually talk to filebeat anymore? But then again, I am using file inputs instead.

Hope that solves the issue.

Artur