Azure日志分析不会在Azure Kubernetes服务上检索NGINX入口的日志

Question

I am new to Kubernetes and NGINX Ingress on Microsoft Azure. I got an issue about analyzing log of NGINX Ingress.

Here is the log in my NGINX Ingress pods:

duc@Azure:~$ kubectl logs ducphuongkhang-ingress-nginx-ingress-controller-869b8b966-877bq -n kube-system | grep 'lua'

2018/11/06 16:36:55 [warn] 961#961: *10059 [lua] log.lua:52: {"timestamp":1541522215,"method":"GET","uri":"\/vulnerabilities\/sqli\/","id":"dba39b7d7dc8646b779e","client":"10.244.0.1","alerts":[{"match":1,"msg":"SQL String Termination","id":41003},{"match":1,"msg":"SQL probing attempt","id":41032},{"logdata":8,"match":8,"msg":"Request score greater than score threshold","id":99001}]} while logging request, client: 10.244.0.1, server: dvwa.thesis.analyticsvn.com, request: "GET /vulnerabilities/sqli/?id=%27&Submit=Submit HTTP/2.0", host: "dvwa.thesis.analyticsvn.com", referrer: "https://dvwa.thesis.analyticsvn.com/vulnerabilities/sqli/?id=1%3D1&Submit=Submit"
2018/11/06 16:37:02 [warn] 961#961: *10059 [lua] log.lua:52: {"timestamp":1541522222,"method":"GET","uri":"\/vulnerabilities\/sqli\/","id":"4ac4e0dfe317dcd86346","client":"10.244.0.1","alerts":[{"match":1,"msg":"SQL String Termination","id":41003},{"match":1,"msg":"SQL probing attempt","id":41032},{"logdata":8,"match":8,"msg":"Request score greater than score threshold","id":99001}]} while logging request, client: 10.244.0.1, server: dvwa.thesis.analyticsvn.com, request: "GET /vulnerabilities/sqli/?id=%27&Submit=Submit HTTP/2.0", host: "dvwa.thesis.analyticsvn.com", referrer: "https://dvwa.thesis.analyticsvn.com/vulnerabilities/sqli/?id=1%3D1&Submit=Submit"
2018/11/06 16:37:02 [warn] 961#961: *10059 [lua] log.lua:52: {"timestamp":1541522222,"method":"GET","uri":"\/vulnerabilities\/sqli\/","id":"d0eae7d54dc99773ecc0","client":"10.244.0.1","alerts":[{"match":1,"msg":"SQL String Termination","id":41003},{"match":1,"msg":"SQL probing attempt","id":41032},{"logdata":8,"match":8,"msg":"Request score greater than score threshold","id":99001}]} while logging request, client: 10.244.0.1, server: dvwa.thesis.analyticsvn.com, request: "GET /vulnerabilities/sqli/?id=%27&Submit=Submit HTTP/2.0", host: "dvwa.thesis.analyticsvn.com", referrer: "https://dvwa.thesis.analyticsvn.com/vulnerabilities/sqli/?id=1%3D1&Submit=Submit"
2018/11/06 16:37:03 [warn] 961#961: *10059 [lua] log.lua:52: {"timestamp":1541522223,"method":"GET","uri":"\/vulnerabilities\/sqli\/","id":"be18d7e7800e86789d5d","client":"10.244.0.1","alerts":[{"match":1,"msg":"SQL String Termination","id":41003},{"match":1,"msg":"SQL probing attempt","id":41032},{"logdata":8,"match":8,"msg":"Request score greater than score threshold","id":99001}]} while logging request, client: 10.244.0.1, server: dvwa.thesis.analyticsvn.com, request: "GET /vulnerabilities/sqli/?id=%27&Submit=Submit HTTP/2.0", host: "dvwa.thesis.analyticsvn.com", referrer: "https://dvwa.thesis.analyticsvn.com/vulnerabilities/sqli/?id=1%3D1&Submit=Submit"

And here is my Azure Analytics Log query, which return no value:

ContainerLog | where LogEntry contains "lua"

I want to collect logs that generated by NGINX Ingress (with lua-resty-waf enable) with Azure Log Analytics. Please help me to get there.

Thank you.

Answer 1

After doing the research, I found out that the Azure Log Analytics with OMS Agent in Kubernetes don't support to collect data in the namespace 'kube-system'. Deploy the Ingress in another namespace will allow the Log Analytics to collect log.

Reference: https://github.com/Azure/AKS/issues/293

$ kubectl describe deployments omsagent-rs -n kube-system
Pod Template:
  Labels:           rsName=omsagent-rs
  Annotations:      agentVersion=1.6.0-42
                    dockerProviderVersion=2.0.0-3
  Service Account:  omsagent
    Environment:
      DISABLE_KUBE_SYSTEM_LOG_COLLECTION:  true