无法将Logstash中的json数组插入mongoDB

Question

I'm using Logstash 1.5.2
trying to insert datas in MongoDB with logstash-output-mongodb
It works well only when my parsed logs doesn't contain arrays, when the JSON contains array it generates an exception.

Failed to send event to MongoDB
undefined methodbson_type
NoMethodError: undefined method `error_code' for #NoMethodError:0x39794292

@cancelled=false>, :exception=>#<NoMethodError: undefined method bson_type' for "2015-08-24T12:26:19.664Z":LogStash::Timestamp>, :backtrace=>["/opt/logstash/vendor/bundle/jruby/1.9/gems/bson-3.2.1-java/lib/bson/hash.rb:44:in to_bson'", "org/jruby/RubyHash.java:1341:in each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/bson-3.2.1-java/lib/bson/hash.rb:43:in to_bson'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/bson-3.2.1-java/lib/bson/encodable.rb:57:in encode_with_placeholder_and_null'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/bson-3.2.1-java/lib/bson/hash.rb:42:in to_bson'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/bson-3.2.1-java/lib/bson/array.rb:49:in to_bson'", "org/jruby/RubyArray.java:1613:in each'", "org/jruby/RubyEnumerable.java:978:in each_with_index'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/bson-3.2.1-java/lib/bson/array.rb:46:in to_bson'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/bson-3.2.1-java/lib/bson/encodable.rb:57:in encode_with_placeholder_and_null'" , "/opt/logstash/vendor/bundle/jruby/1.9/gems/bson-3.2.1-java/lib/bson/array.rb:45:in to_bson'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/bson-3.2.1-java/lib/bson/hash.rb:46:in to_bson'", "org/jruby/RubyHash.java:1341:in each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/bson-3.2.1-java/lib/bson/hash.rb:43:in to_bson'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/bson-3.2.1-java/lib/bson/encodable.rb:57:in encode_with_placeholder_and_null'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/bson-3.2.1-java/lib/bson/hash.rb:42:in to_bson'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/mongo-2.0.6/lib/mongo/protocol/serializers.rb:155:in serialize'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/mongo-2.0.6/lib/mongo/protocol/message.rb:153:in serialize_fields'", "org/jruby/RubyArray.java:1613:in each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/mongo-2.0.6/lib/mongo/protocol/message.rb:141:in serialize_fields'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/mongo-2.0.6/lib/mongo/protocol/mes sage.rb:70:in serialize'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/mongo-2.0.6/lib/mongo/server/connection.rb:123:in write'", "org/jruby/RubyArray.java:1613:in each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/mongo-2.0.6/lib/mongo/server/connection.rb:122:in write'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/mongo-2.0.6/lib/mongo/server/connectable.rb:66:in dispatch'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/mongo-2.0.6/lib/mongo/loggable.rb:44:in log'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/mongo-2.0.6/lib/mongo/loggable.rb:67:in log_debug'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/mongo-2.0.6/lib/mongo/server/connectable.rb:65:in dispatch'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/mongo-2.0.6/lib/mongo/operation/executable.rb:35:in execute'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/mongo-2.0.6/lib/mongo/server/connection_pool.rb:99:in with_connection'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/mongo-2.0.6/lib/mongo/server/context.rb:63:in with_connection'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/mongo-2.0.6/lib/mongo/operation/executable.rb:34:in with_connection'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/mongo-2.0.6/lib/mongo/operation/executable.rb:34:in execute'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/mongo-2.0.6/lib/mongo/operation/write/insert.rb:72:in execute_write_command'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/mongo-2.0.6/lib/mongo/operation/write/insert.rb:62:in execute'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/mongo-2.0.6/lib/mongo/collection.rb:190:in insert_many'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/mongo-2.0.6/lib/mongo/collection.rb:175:in insert_one'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-mongodb-0.1.4/lib/logstash/outputs/mongodb.rb:56:in receive'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.2.2-java/lib/logstash/outputs/base.rb:88:in handle'", "(eval):513:in output_func'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.2.2-java/lib/logstash/pipeline.rb:243:in outputworker'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5. 2.2-java/lib/logstash/pipeline.rb:165:in start_outputs'"], :level=>:warn} NoMethodError: undefined method error_code' for # receive at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-mongodb-0.1.4/lib/logstash/outputs/mongodb.rb:60 handle at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.2.2-java/lib/logstash/outputs/base.rb:88 output_func at (eval):513 outputworker at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.2.2-java/lib/logstash/pipeline strong text .rb:243 start_outputs at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.2.2-java/lib/logstash/pipeline.rb:165 `

Any idea to solve this issue please ? Thanks !

Answer 1

I have this same error when logstash mongoldb output configuration has

isodate => true

I have no arrays in my data. Just Nginx access-log data. When I output my data with

stdout { codec => rubydebug }

@timestamp looks like this

{
"@timestamp" => "2015-09-16T18:49:11.347Z",
}

Main issues is that when I output this to mongodb, the data is inserted as string with quotes escaped like this:

"@timestamp": "\"2015-09-16T18:14:06.178Z\"",

I'm using logstash 1.5.4 and mongodb 3.0.6, and the error I'm getting is

:exception=>#<NoMethodError: undefined method `bson_type' for "2015-09-16T18:33:52.474Z":LogStash::Timestamp>, :backtrace=>["/opt/logstash/vendor/bundle/jruby/1.9/gems/bson-3.2.4-java/lib/bson/hash.rb:44:in `to_bson'", [...]

I haven't found any solution for this yet.