如何从LOGSTASH创建索引并映射到ES

Question

I've been following this Tutorial for impport data from a DB into LOGSTASh and create a Idex and Mapping into Elastic Search INSERT INTO LOGSTASH SELECT DATA FROM DATABASE

This is my OUTPUT based on my Configurations file:

[2017-10-12T11:50:45,807][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"C:/Users/Bruno/Downloads/logstash-5.6.2/logstash-5.6.2/modules/fb_apache/configuration"}
[2017-10-12T11:50:45,812][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"C:/Users/Bruno/Downloads/logstash-5.6.2/logstash-5.6.2/modules/netflow/configuration"}
[2017-10-12T11:50:46,518][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}}
[2017-10-12T11:50:46,521][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://localhost:9200/, :path=>"/"}
[2017-10-12T11:50:46,652][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://localhost:9200/"}
[2017-10-12T11:50:46,654][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2017-10-12T11:50:46,716][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>50001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"_all"=>{"enabled"=>true, "norms"=>false}, "dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date", "include_in_all"=>false}, "@version"=>{"type"=>"keyword", "include_in_all"=>false}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2017-10-12T11:50:46,734][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost:9200"]}
[2017-10-12T11:50:46,749][INFO ][logstash.pipeline        ] Starting pipeline {"id"=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>500}
[2017-10-12T11:50:47,053][INFO ][logstash.pipeline        ] Pipeline main started
[2017-10-12T11:50:47,196][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2017-10-12T11:50:47,817][INFO ][logstash.inputs.jdbc     ] (0.130000s) SELECT * from EP_RDA_STRING
[2017-10-12T11:50:53,095][WARN ][logstash.agent           ] stopping pipeline {:id=>"main"}

Everything seems OK, at least I think. Except the fact that querying the ES server to OUTPUT indexes and Mappings, I have it Empty.

http://localhost:9200/_all/_mapping

{}

http://localhost:9200/_cat/indices?v

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size

this is my File Config:

input {
    jdbc {
        # sqlserver jdbc connection string to our database, mydb        
        jdbc_connection_string => "jdbc:sqlserver://localhost:1433;databaseName=RDA; integratedSecurity=true;"
        # The user we wish to execute our statement as
        jdbc_user => ""
        # The path to our downloaded jdbc driver
        jdbc_driver_library => "C:\mypath\sqljdbc_6.2\enu\mssql-jdbc-6.2.1.jre8.jar"
        # The name of the driver class for Postgresql
        jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
        # our query
        statement => "SELECT * from EP_RDA_STRING"
    }
}
output {
    elasticsearch {

        index => "RDA"
        document_type => "RDA_string_view"
        document_id => "%{ndb_no}"
        hosts => "localhost:9200"
    }
}

Answer 1

Which version of logstash are you using? What is the command that you are using to start the logstash? Make sure that the input and output blocks resemble the one that is given below

input {
    beats {
        port => "29600"
        type => "weblogic-server"
    }
}
filter {
}

output {
    elasticsearch { 
      hosts => ["127.0.0.1:9200"] 
      index => "logstash-%{+YYYY.MM.dd}"
    }
    stdout { codec => rubydebug }
}