NetData: Even after enabling plugin web_log netdata says disabled in error log and not shows up in UI

Question

I have netdata setup on the same system as nginx. I am monitoring stub_status already through netdata and want to monitor the access.log log file as well.

I first copied /usr/lib/netdata/conf.d/python.d/web_log.conf strong text in directory /etc/netadata/plugin.d/ as this is the expected path as mentioned in documentation.

Then I enabled the plugin in file /usr/lib/netdata/conf.d/python.d.conf , but even then i cannot see the section in dashboard and see in netdata's error.log that

Even after enablng web_log plugin in file /usr/lib/neitdata/conf.d/python.d.conf and /usr/lib/netdata/conf.d/go.d.conf just to be sure, i can see in error.log that its disabled

2019-05-04 21:05:55: python.d INFO: plugin[main] : using python v2
2019-05-04 21:05:55: python.d INFO: plugin[main] : starting setup
2019-05-04 21:05:55: python.d INFO: plugin[main] : checking for config in ['/etc/netdata', '/usr/lib/netdata/conf.d']
2019-05-04 21:05:55: python.d INFO: plugin[main] : config found, loading config '/etc/netdata/python.d.conf'
2019-05-04 21:05:55: python.d INFO: plugin[main] : config successfully loaded
2019-05-04 21:05:55: python.d INFO: plugin[main] : starting checker process (1 module(s) to check)
2019-05-04 21:05:55: python.d INFO: plugin[checker] : starting...
2019-05-04 21:05:55: python.d INFO: plugin[checker] : web_log : checking
2019-05-04 21:05:55: python.d INFO: plugin[checker] : web_log : source successfully loaded
2019-05-04 21:05:55: python.d INFO: plugin[checker] : web_log : found config file '/etc/netdata/python.d/web_log.conf'
2019-05-04 21:05:55: python.d INFO: plugin[checker] : web_log : created 13 job(s) from the config
2019-05-04 21:05:55: python.d INFO: web_log[nginx_log] : Collected data: ['request', 'code', 'bytes_sent', 'address']
2019-05-04 21:05:55: python.d INFO: plugin[checker] : web_log[nginx_log] : check successful
2019-05-04 21:05:55: python.d ERROR: web_log[nginx_log2] : /var/log/nginx/localhost.access_log not readable or not exist
2019-05-04 21:05:55: python.d INFO: plugin[checker] : web_log[nginx_log2] : check failed, skipping the job
2019-05-04 21:05:55: python.d ERROR: web_log[apache_log] : /var/log/apache2/access.log not readable or not exist
2019-05-04 21:05:55: python.d INFO: plugin[checker] : web_log[apache_log] : check failed, skipping the job
2019-05-04 21:05:55: python.d ERROR: web_log[apache_log2] : /var/log/apache2/access_log not readable or not exist
2019-05-04 21:05:55: python.d INFO: plugin[checker] : web_log[apache_log2] : check failed, skipping the job
2019-05-04 21:05:55: python.d ERROR: web_log[apache_log3] : /var/log/httpd/access_log not readable or not exist
2019-05-04 21:05:55: python.d INFO: plugin[checker] : web_log[apache_log3] : check failed, skipping the job
2019-05-04 21:05:55: python.d ERROR: web_log[apache_vhosts_log] : /var/log/apache2/other_vhosts_access.log not readable or not exist
2019-05-04 21:05:55: python.d INFO: plugin[checker] : web_log[apache_vhosts_log] : check failed, skipping the job
2019-05-04 21:05:55: python.d ERROR: web_log[gunicorn_log] : /var/log/gunicorn/access.log not readable or not exist
2019-05-04 21:05:55: python.d INFO: plugin[checker] : web_log[gunicorn_log] : check failed, skipping the job
2019-05-04 21:05:55: python.d ERROR: web_log[gunicorn_log2] : /var/log/gunicorn/gunicorn-access.log not readable or not exist
2019-05-04 21:05:55: python.d INFO: plugin[checker] : web_log[gunicorn_log2] : check failed, skipping the job
2019-05-04 21:05:55: python.d ERROR: web_log[apache_cache] : /var/log/apache/cache.log not readable or not exist
2019-05-04 21:05:55: python.d INFO: plugin[checker] : web_log[apache_cache] : check failed, skipping the job
2019-05-04 21:05:55: python.d ERROR: web_log[apache2_cache] : /var/log/apache2/cache.log not readable or not exist
2019-05-04 21:05:55: python.d INFO: plugin[checker] : web_log[apache2_cache] : check failed, skipping the job
2019-05-04 21:05:55: python.d ERROR: web_log[httpd_cache] : /var/log/httpd/cache.log not readable or not exist
2019-05-04 21:05:55: python.d INFO: plugin[checker] : web_log[httpd_cache] : check failed, skipping the job
2019-05-04 21:05:55: python.d ERROR: web_log[squid_log1] : /var/log/squid3/access.log not readable or not exist
2019-05-04 21:05:55: python.d INFO: plugin[checker] : web_log[squid_log1] : check failed, skipping the job
2019-05-04 21:05:55: python.d ERROR: web_log[squid_log2] : /var/log/squid/access.log not readable or not exist
2019-05-04 21:05:55: python.d INFO: plugin[checker] : web_log[squid_log2] : check failed, skipping the job
2019-05-04 21:05:55: python.d INFO: plugin[checker] : terminating...
2019-05-04 21:05:55: python.d INFO: plugin[main] : stopping checker process
2019-05-04 21:05:55: python.d INFO: plugin[main] : setup complete, 1 active module(s) : '['web_log']'
2019-05-04 21:05:55: python.d INFO: plugin[main] : web_log : created 1 job(s)
2019-05-04 21:05:55: python.d INFO: plugin[main] : web_log[nginx_log] : init successful
2019-05-04 21:05:55: python.d INFO: web_log[nginx_log] : Collected data: ['request', 'code', 'bytes_sent', 'address']
2019-05-04 21:05:55: python.d INFO: plugin[main] : web_log[nginx_log] : check successful
CHART netdata.runtime_web_log_nginx '' 'Execution time for web_log_nginx' 'ms' 'python.d' netdata.pythond_runtime line 145000 1
DIMENSION run_time 'run time' absolute 1 1

2019-05-04 21:05:55: python.d INFO: plugin[main] : web_log[nginx_log] : started in thread
2019-05-04 21:05:55: python.d DEBUG: web_log[nginx_log] : started, update frequency: 1
CHART web_log_nginx.requests_per_ipproto '' 'Requests Per IP Protocol' 'requests/s' 'ip protocols' 'web_log.requests_per_ipproto' stacked 60007 1 '' 'python.d.plugin' 'web_log'
DIMENSION 'req_ipv4' 'ipv4' incremental 1 1 ' '
DIMENSION 'req_ipv6' 'ipv6' incremental 1 1 ' '

BEGIN web_log_nginx.requests_per_ipproto 0
SET 'req_ipv4' = 0
SET 'req_ipv6' = 0
END

CHART web_log_nginx.clients_all '' 'All Time Unique Client IPs' 'unique ips' 'clients' 'web_log.clients_all' stacked 60009 1 '' 'python.d.plugin' 'web_log'
DIMENSION 'unique_tot_ipv4' 'ipv4' absolute 1 1 ' '
DIMENSION 'unique_tot_ipv6' 'ipv6' absolute 1 1 ' '

BEGIN web_log_nginx.clients_all 0
SET 'unique_tot_ipv4' = 0
SET 'unique_tot_ipv6' = 0
END

CHART web_log_nginx.response_statuses '' 'Response Statuses' 'requests/s' 'responses' 'web_log.response_statuses' stacked 60000 1 '' 'python.d.plugin' 'web_log'
DIMENSION 'successful_requests' 'success' incremental 1 1 ' '
DIMENSION 'server_errors' 'error' incremental 1 1 ' '
DIMENSION 'redirects' 'redirect' incremental 1 1 ' '
DIMENSION 'bad_requests' 'bad' incremental 1 1 ' '
DIMENSION 'other_requests' 'other' incremental 1 1 ' '

BEGIN web_log_nginx.response_statuses 0
SET 'successful_requests' = 0
SET 'server_errors' = 0
SET 'redirects' = 0
SET 'bad_requests' = 0
SET 'other_requests' = 0
END

CHART web_log_nginx.response_codes '' 'Response Codes' 'requests/s' 'responses' 'web_log.response_codes' stacked 60001 1 '' 'python.d.plugin' 'web_log'
DIMENSION '2xx' '2xx' incremental 1 1 ' '
DIMENSION '5xx' '5xx' incremental 1 1 ' '
DIMENSION '3xx' '3xx' incremental 1 1 ' '
DIMENSION '4xx' '4xx' incremental 1 1 ' '
DIMENSION '1xx' '1xx' incremental 1 1 ' '
DIMENSION '0xx' 'other' incremental 1 1 ' '
DIMENSION 'unmatched' 'unmatched' incremental 1 1 ' '

BEGIN web_log_nginx.response_codes 0
SET '2xx' = 0
SET '5xx' = 0
SET '3xx' = 0
SET '4xx' = 0
SET '1xx' = 0
SET '0xx' = 0
SET 'unmatched' = 0
END

CHART web_log_nginx.clients '' 'Current Poll Unique Client IPs' 'unique ips' 'clients' 'web_log.clients' stacked 60008 1 '' 'python.d.plugin' 'web_log'
DIMENSION 'unique_cur_ipv4' 'ipv4' incremental 1 1 ' '
DIMENSION 'unique_cur_ipv6' 'ipv6' incremental 1 1 ' '

BEGIN web_log_nginx.clients 0
SET 'unique_cur_ipv4' = 0
SET 'unique_cur_ipv6' = 0
END

CHART web_log_nginx.bandwidth '' 'Bandwidth' 'kilobits/s' 'bandwidth' 'web_log.bandwidth' area 60002 1 '' 'python.d.plugin' 'web_log'
DIMENSION 'resp_length' 'received' incremental 8 1000 ' '
DIMENSION 'bytes_sent' 'sent' incremental -8 1000 ' '

BEGIN web_log_nginx.bandwidth 0
SET 'resp_length' = 0
SET 'bytes_sent' = 0
END

CHART web_log_nginx.http_method '' 'Requests Per HTTP Method' 'requests/s' 'http methods' 'web_log.http_method' stacked 60003 1 '' 'python.d.plugin' 'web_log'
DIMENSION 'GET' 'GET' incremental 1 1 ' '

BEGIN web_log_nginx.http_method 0
SET 'GET' = 0
END

BEGIN netdata.runtime_web_log_nginx 0
SET run_time = 0
END

2019-05-04 21:05:56: python.d DEBUG: web_log[nginx_log] : update => [OK] (elapsed time: 0, failed retries in a row: 0)
BEGIN web_log_nginx.requests_per_ipproto 999766
SET 'req_ipv4' = 0
SET 'req_ipv6' = 0
END

BEGIN web_log_nginx.clients_all 999766
SET 'unique_tot_ipv4' = 0
SET 'unique_tot_ipv6' = 0
END

BEGIN web_log_nginx.response_statuses 999766
SET 'successful_requests' = 0
SET 'server_errors' = 0
SET 'redirects' = 0
SET 'bad_requests' = 0
SET 'other_requests' = 0
END

BEGIN web_log_nginx.response_codes 999766
SET '2xx' = 0
SET '5xx' = 0
SET '3xx' = 0
SET '4xx' = 0
SET '1xx' = 0
SET '0xx' = 0
SET 'unmatched' = 0
END

BEGIN web_log_nginx.clients 999766
SET 'unique_cur_ipv4' = 0
SET 'unique_cur_ipv6' = 0
END

BEGIN web_log_nginx.bandwidth 999766
SET 'resp_length' = 0
SET 'bytes_sent' = 0
END

BEGIN web_log_nginx.http_method 999766
SET 'GET' = 0
END

BEGIN netdata.runtime_web_log_nginx 999766
SET run_time = 0
END

2019-05-04 21:05:57: python.d DEBUG: web_log[nginx_log] : update => [OK] (elapsed time: 0, failed retries in a row: 0)
BEGIN web_log_nginx.requests_per_ipproto 999969
SET 'req_ipv4' = 1
SET 'req_ipv6' = 0
END

BEGIN web_log_nginx.clients_all 999969
SET 'unique_tot_ipv4' = 1
SET 'unique_tot_ipv6' = 0
END

BEGIN web_log_nginx.response_statuses 999969
SET 'successful_requests' = 1
SET 'server_errors' = 0
SET 'redirects' = 0
SET 'bad_requests' = 0
SET 'other_requests' = 0
END

BEGIN web_log_nginx.response_codes 999969
SET '2xx' = 0
SET '5xx' = 0
SET '3xx' = 1
SET '4xx' = 0
SET '1xx' = 0
SET '0xx' = 0
SET 'unmatched' = 0
END

CHART web_log_nginx.http_version '' 'Requests Per HTTP Version' 'requests/s' 'http versions' 'web_log.http_version' stacked 60006 1 '' 'python.d.plugin' 'web_log'
DIMENSION '1_1' '1.1' incremental 1 1 ' '

BEGIN web_log_nginx.http_version 0
SET '1_1' = 1
END

BEGIN web_log_nginx.clients 999969
SET 'unique_cur_ipv4' = 1
SET 'unique_cur_ipv6' = 0
END

CHART web_log_nginx.detailed_response_codes '' 'Detailed Response Codes ' 'requests/s' 'responses' 'web_log.detailed_response_codes' stacked 60010 1 '' 'python.d.plugin' 'web_log'
DIMENSION '304' '304' incremental 1 1 ' '

Answer 1

/var/log/nginx/localhost.access_log not readable or not exist

As the log error says, it's either a problem with the path (path is wrong) or netdata process has problem reading the file. Make sure nginx directory is accessible for netdata user.