简体   繁体   中英

filebeat json logs to elasticsearch max size

I am sending the logs produced by my nodejs app using winston

This is my configuration filebeat.yml

filebeat.inputs:
- type: log
  paths:
    - /var/log/application/*.log
  json.keys_under_root: true

output.elasticsearch:
  hosts: ["elasticsearch.staging:9200"]

setup.kibana:
  host: "kibana.staging:5601"

Logs are written in for example /var/log/application/app-bank.log

I have something like that

{"message":"Application started","level":"info","application":"app-bank","environment":"staging"}
{"message":"Header x-website-name not found","stack":"Error: Header x-website-name not found\n    at context.httpApp.use (/usr/src/app/controller/index.js:30:39)\n    at Layer.handle [as handle_request] (/usr/src/app/node_modules/express/lib/router/layer.js:95:5)\n    at trim_prefix (/usr/src/app/node_modules/express/lib/router/index.js:317:13)\n    at /usr/src/app/node_modules/express/lib/router/index.js:284:7\n    at Function.process_params (/usr/src/app/node_modules/express/lib/router/index.js:335:12)\n    at next (/usr/src/app/node_modules/express/lib/router/index.js:275:10)\n    at context.httpApp.use (/usr/src/app/controller/index.js:25:9)\n    at Layer.handle [as handle_request] (/usr/src/app/node_modules/express/lib/router/layer.js:95:5)\n    at trim_prefix (/usr/src/app/node_modules/express/lib/router/index.js:317:13)\n    at /usr/src/app/node_modules/express/lib/router/index.js:284:7\n    at Function.process_params (/usr/src/app/node_modules/express/lib/router/index.js:335:12)\n    at next (/usr/src/app/node_modules/express/lib/router/index.js:275:10)\n    at session (/usr/src/app/node_modules/express-session/index.js:468:7)\n    at Layer.handle [as handle_request] (/usr/src/app/node_modules/express/lib/router/layer.js:95:5)\n    at trim_prefix (/usr/src/app/node_modules/express/lib/router/index.js:317:13)\n    at /usr/src/app/node_modules/express/lib/router/index.js:284:7","level":"error","application":"app-bank","environment":"staging"}

The first log is received my elasticsearch, but not the second one, I can not see any log, the problem should be because of the stack key I do not know if it because of the format of the stack or the log is passing a certain size.

我修复了它

json.add_error_key: true

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM