简体   繁体   中英

Logstash can't create an index in Elasticsearch

I have logstash and elasticsearch on one box. Everything seems to be working (no obvious errors), but no information is getting to elasticsearch. I have noticed that logstash seems to be unable to create an index in ES:

http://192.168.250.49:9200/_status?pretty=true

{
  "_shards" : {
    "total" : 0,
    "successful" : 0,
    "failed" : 0
  },
  "indices" : { }
}

http://192.168.250.49:9200/_aliases?pretty

{}

logstash console (bin/logstash -f configfile --debug):

{:timestamp=>"2015-10-20T15:25:35.776000-0400", :message=>"Reading config file", :file=>"logstash/agent.rb", :level=>:debug, :line=>"309", :method=>"local_config"}
{:timestamp=>"2015-10-20T15:25:35.894000-0400", :message=>"Compiled pipeline code:\n        @inputs = []\n        @filters = []\n        @outputs = []\n        @periodic_flushers = []\n        @shutdown_flushers = []\n\n          @input_file_1 = plugin(\"input\", \"file\", LogStash::Util.hash_merge_many({ \"type\" => (\"syslog\") }, { \"path\" => [(\"/var/log/python_apps/*.log\"), (\"/var/log/python_apps/*.log.*\")] }))\n\n          @inputs << @input_file_1\n\n          @output_elasticsearch_2 = plugin(\"output\", \"elasticsearch\", LogStash::Util.hash_merge_many({ \"host\" => (\"192.168.250.49\") }, { \"protocol\" => (\"http\") }))\n\n          @outputs << @output_elasticsearch_2\n\n          @output_stdout_3 = plugin(\"output\", \"stdout\", LogStash::Util.hash_merge_many({ \"codec\" => (\"rubydebug\") }))\n\n          @outputs << @output_stdout_3\n\n  def filter_func(event)\n    events = [event]\n    @logger.debug? && @logger.debug(\"filter received\", :event => event.to_hash)\n    events\n  end\n  def output_func(event)\n    @logger.debug? && @logger.debug(\"output received\", :event => event.to_hash)\n    @output_elasticsearch_2.handle(event)\n    @output_stdout_3.handle(event)\n    \n  end", :level=>:debug, :file=>"logstash/pipeline.rb", :line=>"29", :method=>"initialize"}
{:timestamp=>"2015-10-20T15:25:35.907000-0400", :message=>"Plugin not defined in namespace, checking for plugin file", :type=>"input", :name=>"file", :path=>"logstash/inputs/file", :level=>:debug, :file=>"logstash/plugin.rb", :line=>"133", :method=>"lookup"}
{:timestamp=>"2015-10-20T15:25:35.925000-0400", :message=>"Plugin not defined in namespace, checking for plugin file", :type=>"codec", :name=>"plain", :path=>"logstash/codecs/plain", :level=>:debug, :file=>"logstash/plugin.rb", :line=>"133", :method=>"lookup"}
{:timestamp=>"2015-10-20T15:25:35.949000-0400", :message=>"config LogStash::Codecs::Plain/@charset = \"UTF-8\"", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:35.963000-0400", :message=>"config LogStash::Inputs::File/@type = \"syslog\"", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:35.975000-0400", :message=>"config LogStash::Inputs::File/@path = [\"/var/log/python_apps/*.log\", \"/var/log/python_apps/*.log.*\"]", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:35.988000-0400", :message=>"config LogStash::Inputs::File/@debug = false", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:35.991000-0400", :message=>"config LogStash::Inputs::File/@codec = <LogStash::Codecs::Plain charset=>\"UTF-8\">", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.004000-0400", :message=>"config LogStash::Inputs::File/@add_field = {}", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.006000-0400", :message=>"config LogStash::Inputs::File/@stat_interval = 1", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.007000-0400", :message=>"config LogStash::Inputs::File/@discover_interval = 15", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.020000-0400", :message=>"config LogStash::Inputs::File/@sincedb_write_interval = 15", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.022000-0400", :message=>"config LogStash::Inputs::File/@start_position = \"end\"", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.023000-0400", :message=>"config LogStash::Inputs::File/@delimiter = \"\\n\"", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.037000-0400", :message=>"Plugin not defined in namespace, checking for plugin file", :type=>"output", :name=>"elasticsearch", :path=>"logstash/outputs/elasticsearch", :level=>:debug, :file=>"logstash/plugin.rb", :line=>"133", :method=>"lookup"}
{:timestamp=>"2015-10-20T15:25:36.422000-0400", :message=>"config LogStash::Codecs::Plain/@charset = \"UTF-8\"", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.453000-0400", :message=>"config LogStash::Outputs::ElasticSearch/@host = [\"192.168.250.49\"]", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.466000-0400", :message=>"config LogStash::Outputs::ElasticSearch/@protocol = \"http\"", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.480000-0400", :message=>"config LogStash::Outputs::ElasticSearch/@type = \"\"", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.492000-0400", :message=>"config LogStash::Outputs::ElasticSearch/@tags = []", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.515000-0400", :message=>"config LogStash::Outputs::ElasticSearch/@exclude_tags = []", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.528000-0400", :message=>"config LogStash::Outputs::ElasticSearch/@codec = <LogStash::Codecs::Plain charset=>\"UTF-8\">", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.539000-0400", :message=>"config LogStash::Outputs::ElasticSearch/@workers = 1", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.551000-0400", :message=>"config LogStash::Outputs::ElasticSearch/@index = \"logstash-%{+YYYY.MM.dd}\"", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.563000-0400", :message=>"config LogStash::Outputs::ElasticSearch/@manage_template = true", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.586000-0400", :message=>"config LogStash::Outputs::ElasticSearch/@template_name = \"logstash\"", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.588000-0400", :message=>"config LogStash::Outputs::ElasticSearch/@template_overwrite = false", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.611000-0400", :message=>"config LogStash::Outputs::ElasticSearch/@embedded = false", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.623000-0400", :message=>"config LogStash::Outputs::ElasticSearch/@embedded_http_port = \"9200-9300\"", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.635000-0400", :message=>"config LogStash::Outputs::ElasticSearch/@max_inflight_requests = 50", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.644000-0400", :message=>"config LogStash::Outputs::ElasticSearch/@flush_size = 5000", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.651000-0400", :message=>"config LogStash::Outputs::ElasticSearch/@idle_flush_time = 1", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.655000-0400", :message=>"config LogStash::Outputs::ElasticSearch/@action = \"index\"", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.662000-0400", :message=>"config LogStash::Outputs::ElasticSearch/@path = \"/\"", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.667000-0400", :message=>"config LogStash::Outputs::ElasticSearch/@ssl = false", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.672000-0400", :message=>"config LogStash::Outputs::ElasticSearch/@ssl_certificate_verification = true", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.673000-0400", :message=>"config LogStash::Outputs::ElasticSearch/@sniffing = false", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.675000-0400", :message=>"config LogStash::Outputs::ElasticSearch/@max_retries = 3", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.676000-0400", :message=>"config LogStash::Outputs::ElasticSearch/@retry_max_items = 5000", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.689000-0400", :message=>"config LogStash::Outputs::ElasticSearch/@retry_max_interval = 5", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.691000-0400", :message=>"config LogStash::Outputs::ElasticSearch/@doc_as_upsert = false", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.692000-0400", :message=>"config LogStash::Outputs::ElasticSearch/@upsert = \"\"", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.694000-0400", :message=>"Plugin not defined in namespace, checking for plugin file", :type=>"output", :name=>"stdout", :path=>"logstash/outputs/stdout", :level=>:debug, :file=>"logstash/plugin.rb", :line=>"133", :method=>"lookup"}
{:timestamp=>"2015-10-20T15:25:36.754000-0400", :message=>"Plugin not defined in namespace, checking for plugin file", :type=>"codec", :name=>"rubydebug", :path=>"logstash/codecs/rubydebug", :level=>:debug, :file=>"logstash/plugin.rb", :line=>"133", :method=>"lookup"}
{:timestamp=>"2015-10-20T15:25:36.782000-0400", :message=>"config LogStash::Codecs::RubyDebug/@metadata = false", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.797000-0400", :message=>"config LogStash::Outputs::Stdout/@codec = <LogStash::Codecs::RubyDebug metadata=>false>", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.809000-0400", :message=>"config LogStash::Outputs::Stdout/@type = \"\"", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.831000-0400", :message=>"config LogStash::Outputs::Stdout/@tags = []", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.843000-0400", :message=>"config LogStash::Outputs::Stdout/@exclude_tags = []", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:36.860000-0400", :message=>"config LogStash::Outputs::Stdout/@workers = 1", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"111", :method=>"config_init"}
{:timestamp=>"2015-10-20T15:25:37.114000-0400", :message=>"Registering file input", :path=>["/var/log/python_apps/*.log", "/var/log/python_apps/*.log.*"], :level=>:info, :file=>"logstash/inputs/file.rb", :line=>"77", :method=>"register"}
{:timestamp=>"2015-10-20T15:25:37.127000-0400", :message=>"No sincedb_path set, generating one based on the file path", :sincedb_path=>"/root/.sincedb_8f309eb34476af59efaabf28f6aac73a", :path=>["/var/log/python_apps/*.log", "/var/log/python_apps/*.log.*"], :level=>:info, :file=>"logstash/inputs/file.rb", :line=>"120", :method=>"register"}
{:timestamp=>"2015-10-20T15:25:37.141000-0400", :message=>"Normalizing http path", :path=>"/", :normalized=>"/", :level=>:debug, :file=>"logstash/outputs/elasticsearch.rb", :line=>"353", :method=>"register"}
{:timestamp=>"2015-10-20T15:25:37.148000-0400", :message=>"Create client to elasticsearch server on 192.168.250.49:", :level=>:info, :file=>"logstash/outputs/elasticsearch.rb", :line=>"440", :method=>"register"}
{:timestamp=>"2015-10-20T15:25:37.150000-0400", :message=>"_sincedb_open: reading from /root/.sincedb_8f309eb34476af59efaabf28f6aac73a", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"219", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.193000-0400", :message=>"_sincedb_open: setting [\"10131811\", 0, 64768] to 776284", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.208000-0400", :message=>"_sincedb_open: setting [\"10131826\", 0, 64768] to 1445796", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.210000-0400", :message=>"_sincedb_open: setting [\"10131834\", 0, 64768] to 8271711", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.221000-0400", :message=>"_sincedb_open: setting [\"10131838\", 0, 64768] to 6486678", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.234000-0400", :message=>"_sincedb_open: setting [\"10131855\", 0, 64768] to 3707417", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.246000-0400", :message=>"_sincedb_open: setting [\"10131860\", 0, 64768] to 82806", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.256000-0400", :message=>"_sincedb_open: setting [\"10131861\", 0, 64768] to 4591241", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.269000-0400", :message=>"_sincedb_open: setting [\"10131865\", 0, 64768] to 0", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.334000-0400", :message=>"_sincedb_open: setting [\"10131866\", 0, 64768] to 0", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.340000-0400", :message=>"_sincedb_open: setting [\"10131869\", 0, 64768] to 0", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.401000-0400", :message=>"_sincedb_open: setting [\"9870144\", 0, 64768] to 0", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.432000-0400", :message=>"_sincedb_open: setting [\"10068312\", 0, 64768] to 10485833", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.445000-0400", :message=>"_sincedb_open: setting [\"10068313\", 0, 64768] to 10486125", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.454000-0400", :message=>"_sincedb_open: setting [\"10068314\", 0, 64768] to 10485833", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.466000-0400", :message=>"_sincedb_open: setting [\"10077159\", 0, 64768] to 10486142", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.468000-0400", :message=>"_sincedb_open: setting [\"10131808\", 0, 64768] to 10485990", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.480000-0400", :message=>"_sincedb_open: setting [\"10131809\", 0, 64768] to 10485800", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.512000-0400", :message=>"_sincedb_open: setting [\"10131810\", 0, 64768] to 10485883", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.526000-0400", :message=>"_sincedb_open: setting [\"10131812\", 0, 64768] to 10485802", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.539000-0400", :message=>"_sincedb_open: setting [\"10131813\", 0, 64768] to 10485915", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.555000-0400", :message=>"_sincedb_open: setting [\"10131814\", 0, 64768] to 10486067", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.577000-0400", :message=>"_sincedb_open: setting [\"10131815\", 0, 64768] to 10485843", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.588000-0400", :message=>"_sincedb_open: setting [\"10131816\", 0, 64768] to 10485762", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.602000-0400", :message=>"_sincedb_open: setting [\"10131817\", 0, 64768] to 10485831", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.663000-0400", :message=>"_sincedb_open: setting [\"10131818\", 0, 64768] to 10485945", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.666000-0400", :message=>"_sincedb_open: setting [\"10131819\", 0, 64768] to 10485872", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.673000-0400", :message=>"_sincedb_open: setting [\"10131820\", 0, 64768] to 10485787", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.699000-0400", :message=>"_sincedb_open: setting [\"10131821\", 0, 64768] to 10485819", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.714000-0400", :message=>"_sincedb_open: setting [\"10131822\", 0, 64768] to 10486265", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.730000-0400", :message=>"_sincedb_open: setting [\"10131823\", 0, 64768] to 10485792", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.741000-0400", :message=>"_sincedb_open: setting [\"10131824\", 0, 64768] to 10485982", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.744000-0400", :message=>"_sincedb_open: setting [\"10131825\", 0, 64768] to 10485807", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.782000-0400", :message=>"_sincedb_open: setting [\"10131827\", 0, 64768] to 10486007", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.802000-0400", :message=>"_sincedb_open: setting [\"10131828\", 0, 64768] to 10485770", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.815000-0400", :message=>"_sincedb_open: setting [\"10131829\", 0, 64768] to 10492229", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.860000-0400", :message=>"_sincedb_open: setting [\"10131830\", 0, 64768] to 10485793", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.875000-0400", :message=>"_sincedb_open: setting [\"10131831\", 0, 64768] to 10485828", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.887000-0400", :message=>"_sincedb_open: setting [\"10131832\", 0, 64768] to 10485773", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.899000-0400", :message=>"_sincedb_open: setting [\"10131833\", 0, 64768] to 10485845", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.911000-0400", :message=>"_sincedb_open: setting [\"10131835\", 0, 64768] to 10485965", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.923000-0400", :message=>"_sincedb_open: setting [\"10131836\", 0, 64768] to 10485881", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.935000-0400", :message=>"_sincedb_open: setting [\"10131837\", 0, 64768] to 10485809", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.860000-0400", :message=>"Automatic template management enabled", :manage_template=>"true", :level=>:info, :file=>"logstash/outputs/elasticsearch.rb", :line=>"448", :method=>"register"}
{:timestamp=>"2015-10-20T15:25:37.953000-0400", :message=>"_sincedb_open: setting [\"10131839\", 0, 64768] to 10485806", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.987000-0400", :message=>"_sincedb_open: setting [\"10131840\", 0, 64768] to 10485819", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:37.994000-0400", :message=>"_sincedb_open: setting [\"10131841\", 0, 64768] to 10485820", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:38.019000-0400", :message=>"_sincedb_open: setting [\"10131842\", 0, 64768] to 10486363", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}
{:timestamp=>"2015-10-20T15:25:38.021000-0400", :message=>"_sincedb_open: setting [\"10131843\", 0, 64768] to 10485815", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"223", :method=>"_sincedb_open"}

(If you need more console data, please tell me. @ char limit)

logstash conf:

input {
  file {
    type => "syslog"
    path => ["/var/log/python_apps/*.log", "/var/log/python_apps/*.log.*"]
  }
}

output {
  elasticsearch {
              host => "192.138.250.49"
              protocol => "http"
              cluster => "elastic"
             }
  stdout {
       codec => rubydebug
      }
}

logstash err is empty

logstash log is empty

logstash.stdout: Sending logstash logs to /var/log/logstash/logstash.log.

ES.log (only file in /var/log/elasticsearch with data):

[2015-10-20 15:37:35,031][INFO ][node                     ] [Harald Jaekelsson] version[1.4.4], pid[2573], build[c88f77f/2015-02-19T13:05:36Z]
[2015-10-20 15:37:35,032][INFO ][node                     ] [Harald Jaekelsson] initializing ...
[2015-10-20 15:37:35,034][INFO ][plugins                  ] [Harald Jaekelsson] loaded [], sites []
[2015-10-20 15:37:37,484][INFO ][node                     ] [Harald Jaekelsson] initialized
[2015-10-20 15:37:37,484][INFO ][node                     ] [Harald Jaekelsson] starting ...
[2015-10-20 15:37:37,591][INFO ][transport                ] [Harald Jaekelsson] bound_address {inet[/192.168.250.49:9300]}, publish_address {inet[/192.168.250.49:9300]}
[2015-10-20 15:37:37,607][INFO ][discovery                ] [Harald Jaekelsson] elasticeagle/3_vjAMdtR4C7oiTgr03wng
[2015-10-20 15:37:41,370][INFO ][cluster.service          ] [Harald Jaekelsson] new_master [Harald Jaekelsson][3_vjAMdtR4C7oiTgr03wng][localhost.localdomain][inet[/192.168.250.49:9300]], reason: zen-disco-join (elected_as_master)
[2015-10-20 15:37:41,512][INFO ][gateway                  ] [Harald Jaekelsson] recovered [0] indices into cluster_state
[2015-10-20 15:37:41,528][INFO ][http                     ] [Harald Jaekelsson] bound_address {inet[/192.168.250.49:9200]}, publish_address {inet[/192.168.250.49:9200]}
[2015-10-20 15:37:41,528][INFO ][node                     ] [Harald Jaekelsson] started

UPDATE: Things work if I start logstash manually. People have inferred that because this works, logstash isn't acting under user/group logstash during init. But, etc/init.d/logstash reveals that it's working as logstash.

People have also told me .sincedb could be the cause of some of this. But, I can't find it on my system, even though it's referenced by logstash and it claims such is in the /root folder.

{:timestamp=>"2015-10-21T08:57:56.694000-0400", :message=>"No sincedb_path set, generating one based on the file path", :sincedb_path=>"/root/.sincedb_8f309eb34476af59efaabf28f6aac73a", :path=>["/var/log/python_apps/*.log", "/var/log/python_apps/*.log.*"], :level=>:info, :file=>"logstash/inputs/file.rb", :line=>"120", :method=>"register"}

Discussion in the link may help : https://discuss.elastic.co/t/logstash-doesnt-create-index-in-es-no-errors/32617

It, says

"The reason it worked when you ran Logstash by hand was that sincedb state isn't shared between users so the files were treated as brand new and were processed from the begininning. Again, if you want to reprocess log files you have to clear the sincedb entries. If you do that it'll work even when you run Logstash as a service." You can find sincedb files in logstash log, provided that you are running in debug mode.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM