无法访问统计信息 - WSO2 API Manager和WSO2 BAM

Question

I have installed this man times now only to get the same issues. Upon clicking stats in the publisher / store i get referred to the wiki and in the carbon i get the errors below -

TID[-1234] [AM] [2014-07-01 15:42:26,892] ERROR {org.wso2.carbon.apimgt.hostobjects.APIProviderHostObject} - Error while invoking APIUsageStatisticsClient for ProviderAPIUsage org.wso2.carbon.apimgt.usage.client.APIUsageStatisticsClient.(APIUsageStatisticsClient.java:115) org.wso2.carbon.apimgt.hostobjects.APIProviderHostObject.jsFunction_getFirstAccessTime(APIProviderHostObject.java:3937) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:606) org.mozilla.javascript.MemberBox.invoke(MemberBox.java:126) org.mozilla.javascript.FunctionObject.call(FunctionObject.java:386) org.mozilla.javascript.optimizer.OptRuntime.call2(OptRuntime.java:42) org.jaggeryjs.rhino.publisher.modules.statistics.c1._c_getFirstAccessTime_14(/publisher/modules/statistics/usage.jag:382) org.jaggeryjs. rhino.publisher.modules.statistics.c1.call(/publisher/modules/statistics/usage.jag) org.mozilla.javascript.ScriptRuntime.applyOrCall(ScriptRuntime.java:2430) org.mozilla.javascript.BaseFunction.execIdCall(BaseFunction.java:269) org.mozilla.javascript.IdFunctionObject.call(IdFunctionObject.java:97) org.mozilla.javascript.optimizer.OptRuntime.call2(OptRuntime.java:42) org.jaggeryjs.rhino.publisher.modules.statistics.c0._c_anonymous_14(/publisher/modules/statistics/module.jag:31) org.jaggeryjs.rhino.publisher.modules.statistics.c0.call(/publisher/modules/statistics/module.jag) org.mozilla.javascript.optimizer.OptRuntime.call1(OptRuntime.java:32) org.jaggeryjs.rhino.publisher.site.blocks.stats.api_subscriptions.ajax.c0._c_anonymous_1(/publisher/site/blocks/stats/api-subscriptions/ajax/stats.jag:47) org.jaggeryjs.rhino.publisher.site.blocks.stats.api_subscriptions.ajax.c0.call(/publisher/site/blocks/stats/api-subscriptions/ajax/stats.jag) org.mozilla.javascript.optimizer.OptRuntime.call0(Op tRuntime.java:23) org.jaggeryjs.rhino.publisher.site.blocks.stats.api_subscriptions.ajax.c0._c_script_0(/publisher/site/blocks/stats/api-subscriptions/ajax/stats.jag:4) org.jaggeryjs.rhino.publisher.site.blocks.stats.api_subscriptions.ajax.c0.call(/publisher/site/blocks/stats/api-subscriptions/ajax/stats.jag) org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:394) org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3091) org.jaggeryjs.rhino.publisher.site.blocks.stats.api_subscriptions.ajax.c0.call(/publisher/site/blocks/stats/api-subscriptions/ajax/stats.jag) org.jaggeryjs.rhino.publisher.site.blocks.stats.api_subscriptions.ajax.c0.exec(/publisher/site/blocks/stats/api-subscriptions/ajax/stats.jag) org.jaggeryjs.scriptengine.engine.RhinoEngine.execScript(RhinoEngine.java:570) org.jaggeryjs.scriptengine.engine.RhinoEngine.exec(RhinoEngine.java:273) org.jaggeryjs.jaggery.core.manager.WebAppManager.execute(WebAppManager.java:432) org.jaggeryjs.jaggery.core. JaggeryServlet.doPost(JaggeryServlet.java:29) javax.servlet.http.HttpServlet.service(HttpServlet.java:755) javax.servlet.http.HttpServlet.service(HttpServlet.java:848) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:749) org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:487) org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:379) org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:339) org.jaggeryjs.jaggery.core.JaggeryFilter.doFilter(JaggeryFilter.java:21) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) org.apache.catalina.core.St andardWrapperValve.invoke(StandardWrapperValve.java:222) org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472) org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99) org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178) org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47) org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56) org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47) org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141) org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156) org.apache.catalina.valves.AccessLogValve.invoke(Access LogValve.java:936) org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52) org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407) org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004) org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589) org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:744)

FOLLOWED BY

TID[-1234] [AM] [2014-07-01 15:42:26,985] ERROR {JAGGERY.modules.statistics.usage:jag} - java.lang.NullPointerException: null

Can any one help please?

Thanks,

Ash.

UPDATE -

After working out it is an issue with API-M reading the RDBMS (Thanks to Nadeesha), I have got this log from the API-M console after attempting to access the publisher stats

[2014-07-02 10:26:03,279] ERROR - APIProviderHostObject Error while invoking APIUsageStatisticsClient for ProviderAPIUsage org.wso2.carbon.apimgt.usage.client.exception.APIMgtUsageQueryServiceClientException: Exception while instantiating API manager core objects at org.wso2.carbon.apimgt.usage.client.APIUsageStatisticsClient.(APIUsageStatisticsClient.java:115) at org.wso2.carbon.apimgt.hostobjects.APIProviderHostObject.jsFunction_getFirstAccessTime(APIProviderHostObject.java:3937) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:126) at org.mozilla.javascript.FunctionObject.call(FunctionObject.java:386) at org.mozilla.javascript.optimizer.OptRuntime.call2(OptRuntime.java:42) at org.jaggeryjs.rhin o.publisher.modules.statistics.c1._c_getFirstAccessTime_14(/publisher/modules/statistics/usage.jag:382) at org.jaggeryjs.rhino.publisher.modules.statistics.c1.call(/publisher/modules/statistics/usage.jag) at org.mozilla.javascript.ScriptRuntime.applyOrCall(ScriptRuntime.java:2430) at org.mozilla.javascript.BaseFunction.execIdCall(BaseFunction.java:269) at org.mozilla.javascript.IdFunctionObject.call(IdFunctionObject.java:97) at org.mozilla.javascript.optimizer.OptRuntime.call2(OptRuntime.java:42) at org.jaggeryjs.rhino.publisher.modules.statistics.c0._c_anonymous_14(/publisher/modules/statistics/module.jag:31) at org.jaggeryjs.rhino.publisher.modules.statistics.c0.call(/publisher/modules/statistics/module.jag) at org.mozilla.javascript.optimizer.OptRuntime.call1(OptRuntime.java:32) at org.jaggeryjs.rhino.publisher.site.blocks.stats.api_subscriptions.ajax.c0._c_anonymous_1(/publisher/site/blocks/stats/api-subscriptions/aja x/stats.jag:47) at org.jaggeryjs.rhino.publisher.site.blocks.sta ts.api_subscriptions.ajax.c0.call(/publisher/site/blocks/stats/api-subscriptions/ajax/stats.ja g) at org.mozilla.javascript.optimizer.OptRuntime.call0(OptRuntime.java:23) at org.jaggeryjs.rhino.publisher.site.blocks.stats.api_subscriptions.ajax.c0._c_script_0(/publisher/site/blocks/stats/api-subscriptions/ajax/s tats.jag:4) at org.jaggeryjs.rhino.publisher.site.blocks.stats.api_subscriptions.ajax.c0.call(/publisher/site/blocks/stats/api-subscriptions/ajax/stats.ja g) at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:394) at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3091) at org.jaggeryjs.rhino.publisher.site.blocks.stats.api_subscriptions.ajax.c0.call(/publisher/site/blocks/stats/api-subscriptions/ajax/stats.ja g) at org.jaggeryjs.rhino.publisher.site.blocks.stats.api_subscriptions.ajax.c0.exec(/publisher/site/blocks/stats/api-subscriptions/ajax/stats.ja g) at org.jaggeryjs.scriptengine.engine.RhinoEngine.execScript(RhinoEngine.java:570) at org.jaggeryjs.scriptengine.engine.RhinoEngine.exec(RhinoEngine.java:273) at org.jaggeryjs.jaggery.core.manager.WebAppManager.execute(WebAppManager.java:432) at org.jaggeryjs.jaggery.core.JaggeryServlet.doPost(JaggeryServlet.java:29) at javax.servlet.http.HttpServlet.service(HttpServlet.java:755) at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:749) at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:487) at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:379) at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:339) at org.jaggeryjs.jaggery.core.JaggeryFilter.doFilter(JaggeryFilter.java:21) a t org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99) at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178) at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47) at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56) at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47) at o rg.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141) at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936) at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lan g.Thread.run(Thread.java:744) Caused by: java.io.FileNotFoundException: C:\\WSO2AM~1.0\\bin..\\repository\\conf\\billing-conf.xml (The system cannot find the file specified) at java.io.FileInputStream.open(Native Method) at java.io.FileInputStream.(FileInputStream.java:146) at java.io.FileInputStream.(FileInputStream.java:101) at org.wso2.carbon.apimgt.usage.client.APIUsageStatisticsClient.(APIUsageStatisticsClient.java:106) ... 62 more [2014-07-02 10:26:03,404] ERROR - usage:jag java.lang.NullPointerException: null

Answer 1

There are few ways to debug whats going wrong. How API-M and BAM integration works is that

1. API -> publish stats to BAM

2. BAM stores the stats in Cassandra

3. BAM uses Hive scripts to analyse the data and write the results to a RDBMS mentioned in master-datasources.xml

I would recommend you to check the number 2. by checking if stats are published to BAM. You can do this by login into the management console of BAM and going to the Analytic section and adding the following script

select * from  APIRequestData;

Execute this script and see if any data is returned. If no data is returned then data is not correctly published from API-M to BAM. If data is available here then there is an issue in reading the data from RDBMS to API-M.

Answer 2

Woohoo i fixed the problem! Thanks to Nadeesha for pointing out that API-M wasnt reading the RDBMS data properly (See here to check that - https://stackoverflow.com/a/24523906/84546 )

I then found a mysterious billing-conf.xml not found message in the WSO2 console log, after a little research change the tag inside repository/conf/api-manager.xml from true to false

<!--
        Enable/Disable Usage metering and billing for api usage
     -->
     <EnableBillingAndUsage>**false**</EnableBillingAndUsage>

Voila! It started to work!

Thanks,

Ash.