Logstash API configuration http

Question

Im trying to create an API connection in Logstash and push the data to elasticsearch. Both Elasticsearch and Logstash versions are 7.1.0. Below id the logstash file:

input {
          http_poller {
            urls => {
              test2 => {
                method => get
        user => "readonly"
            password => "mypass@123"
                url => "https://nfr.saas.appdynamics.com/controller/rest/applications?output=JSON
                headers => {
                  Accept => "application/json"
                }
             }
            }
            request_timeout => 60
            # Supports "cron", "every", "at" and "in" schedules by rufus scheduler
            schedule => { cron => "* * * * * UTC"}
            codec => "json"
            # A hash of request metadata info (timing, response headers, etc.) will be sent here
            metadata_target => "http_poller_metadata"
          }
        }

        output {
          elasticsearch { hosts => ["localhost:9200"] }
          stdout { codec => rubydebug }
        }

I am receiving an error:

An unexpected error occurred! {:error=>java.nio.file.AccessDeniedException: D:\logstash-7.1.0\logsta
        sh-7.1.0\data\.lock, :backtrace=>["sun.nio.fs.WindowsException.translateToIOExce
        ption(sun/nio/fs/WindowsException.java:83)", "sun.nio.fs.WindowsException.rethro
        wAsIOException(sun/nio/fs/WindowsException.java:97)", "sun.nio.fs.WindowsException

Edit 1: post giving permissions to data folder suggested by @apt-get_install_skill , below is the timeout error i received:

    [0] "_http_request_failure"
    ],
    "http_request_failure" => {
                "request" => {
             "method" => "get",
                "url" => "https://nfr.saas.appdynamics.com/controller/rest/applications?output=JSON",
            "headers" => {
                "Accept" => "application/json"
            },
               "auth" => {
                 "user" => "readonly",
                 "pass" => "mypass@123",
                "eager" => true
            }
        },
        "runtime_seconds" => 10.004,
                   "name" => "test2",
                  "error" => "connect timed out",
              "backtrace" => nil
    }
}

Im new to APIs, and I'm not sure how to simply fetch the output from the URL. Could you help me on getting this corrected? The URL works when i hit it on my browser.

Answer 1

The problem is not the elasticsearch output but more the permissions of the file

D:\logstash-7.1.0\logstash-7.1.0\data\.lock

as stated in the stacktrace:

error=> java.nio.file.AccessDeniedException: D:\\logstash-7.1.0\\logstash-7.1.0\\data\\.lock , :backtrace=>["sun.nio.fs.WindowsException.translateToIOExce ption(sun/nio/fs/WindowsException.java:83)", "sun.nio.fs.WindowsException.rethro wAsIOException(sun/nio/fs/WindowsException.java:97)", "sun.nio.fs.WindowsException

You need to make sure that the user that executes logstash has the permissions to read and write this file.