WSO2 Identity Server and Analytics don't work properly

Question

I tried to run WSO2 identity server 5.9.0 and Analytics 5.8.0 using docker-compose(I followed this instruction: https://github.com/wso2/docker-is/tree/5.9.x/docker-compose/is-with-analytics )

Once I logged in with admin account at localhost:9443/dashboard, I got the following error.

identity-server_1                      | [2020-10-08 01:18:11,521] [cc08159e-92a6-41ce-a34c-f1ff61a15600] ERROR {org.wso2.carbon.databridge.agent.endpoint.DataEndpointConnectionWorker} - Error while trying to connect to the endpoint. Cannot borrow client for ssl://identity-server-analytics-worker:7712. org.wso2.carbon.databridge.agent.exception.DataEndpointLoginException: Cannot borrow client for ssl://identity-server-analytics-worker:7712.
identity-server_1                      |    at org.wso2.carbon.databridge.agent.endpoint.DataEndpointConnectionWorker.connect(DataEndpointConnectionWorker.java:145)
identity-server_1                      |    at org.wso2.carbon.databridge.agent.endpoint.DataEndpointConnectionWorker.run(DataEndpointConnectionWorker.java:59)
identity-server_1                      |    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
identity-server_1                      |    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
identity-server_1                      |    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
identity-server_1                      |    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
identity-server_1                      |    at java.lang.Thread.run(Thread.java:748)
identity-server_1                      | Caused by: org.wso2.carbon.databridge.agent.exception.DataEndpointLoginException: Error while trying to login to the data receiver.
identity-server_1                      |    at org.wso2.carbon.databridge.agent.endpoint.thrift.ThriftDataEndpoint.login(ThriftDataEndpoint.java:54)
identity-server_1                      |    at org.wso2.carbon.databridge.agent.endpoint.DataEndpointConnectionWorker.connect(DataEndpointConnectionWorker.java:139)
identity-server_1                      |    ... 6 more
identity-server_1                      | Caused by: org.apache.thrift.transport.TTransportException: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
identity-server_1                      |    at org.apache.thrift.transport.TIOStreamTransport.flush(TIOStreamTransport.java:161)
identity-server_1                      |    at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:73)
identity-server_1                      |    at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62)
identity-server_1                      |    at org.wso2.carbon.databridge.commons.thrift.service.secure.ThriftSecureEventTransmissionService$Client.send_connect(ThriftSecureEventTransmissionService.java:104)
identity-server_1                      |    at org.wso2.carbon.databridge.commons.thrift.service.secure.ThriftSecureEventTransmissionService$Client.connect(ThriftSecureEventTransmissionService.java:95)
identity-server_1                      |    at org.wso2.carbon.databridge.agent.endpoint.thrift.ThriftDataEndpoint.login(ThriftDataEndpoint.java:47)
identity-server_1                      |    ... 7 more
identity-server_1                      | Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
identity-server_1                      |    at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
identity-server_1                      |    at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1946)
identity-server_1                      |    at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:316)
identity-server_1                      |    at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310)
identity-server_1                      |    at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639)
identity-server_1                      |    at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223)
identity-server_1                      |    at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
identity-server_1                      |    at sun.security.ssl.Handshaker.process_record(Handshaker.java:965)
identity-server_1                      |    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1064)
identity-server_1                      |    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367)
identity-server_1                      |    at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:750)
identity-server_1                      |    at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123)
identity-server_1                      |    at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
identity-server_1                      |    at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
identity-server_1                      |    at org.apache.thrift.transport.TIOStreamTransport.flush(TIOStreamTransport.java:159)
identity-server_1                      |    ... 12 more
identity-server_1                      | Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
identity-server_1                      |    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397)
identity-server_1                      |    at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302)
identity-server_1                      |    at sun.security.validator.Validator.validate(Validator.java:262)
identity-server_1                      |    at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:330)
identity-server_1                      |    at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:237)
identity-server_1                      |    at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:132)
identity-server_1                      |    at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1621)
identity-server_1                      |    ... 22 more
identity-server_1                      | Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
identity-server_1                      |    at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
identity-server_1                      |    at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
identity-server_1                      |    at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
identity-server_1                      |    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:392)
identity-server_1                      |    ... 28 more
identity-server_1                      | 
identity-server_1                      | [2020-10-08 01:18:16,164] [cc08159e-92a6-41ce-a34c-f1ff61a15600]  WARN {org.apache.thrift.transport.TIOStreamTransport} - Error closing output stream. java.net.SocketException: Socket is closed
identity-server_1                      |    at sun.security.ssl.SSLSocketImpl.checkEOF(SSLSocketImpl.java:1524)
identity-server_1                      |    at sun.security.ssl.SSLSocketImpl.checkWrite(SSLSocketImpl.java:1545)
identity-server_1                      |    at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:71)
identity-server_1                      |    at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
identity-server_1                      |    at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
identity-server_1                      |    at java.io.FilterOutputStream.close(FilterOutputStream.java:158)
identity-server_1                      |    at org.apache.thrift.transport.TIOStreamTransport.close(TIOStreamTransport.java:110)
identity-server_1                      |    at org.apache.thrift.transport.TSocket.close(TSocket.java:235)
identity-server_1                      |    at org.wso2.carbon.databridge.agent.endpoint.thrift.ThriftSecureClientPoolFactory.terminateClient(ThriftSecureClientPoolFactory.java:99)
identity-server_1                      |    at org.wso2.carbon.databridge.agent.client.AbstractClientPoolFactory.destroyObject(AbstractClientPoolFactory.java:71)
identity-server_1                      |    at org.apache.commons.pool.impl.GenericKeyedObjectPool.evict(GenericKeyedObjectPool.java:1976)
identity-server_1                      |    at org.apache.commons.pool.impl.GenericKeyedObjectPool$Evictor.run(GenericKeyedObjectPool.java:2350)
identity-server_1                      |    at java.util.TimerThread.mainLoop(Timer.java:555)
identity-server_1                      |    at java.util.TimerThread.run(Timer.java:505)
identity-server_1                      | 

Also, I got the following error when I enter the analytics dashboard(https://localhost:9643/portal/)

identity-server-analytics-dashboard_1  | [2020-10-08 01:24:22,273] ERROR {org.wso2.transport.http.netty.contractimpl.listener.WebSocketServerHandshakeHandler} - Error during WebSocket server handshake io.netty.handler.codec.DecoderException: javax.net.ssl.SSLException: Received fatal alert: bad_certificate
identity-server-analytics-dashboard_1  |    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:459)
identity-server-analytics-dashboard_1  |    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265)
identity-server-analytics-dashboard_1  |    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
identity-server-analytics-dashboard_1  |    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
identity-server-analytics-dashboard_1  |    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
identity-server-analytics-dashboard_1  |    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359)
identity-server-analytics-dashboard_1  |    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
identity-server-analytics-dashboard_1  |    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
identity-server-analytics-dashboard_1  |    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935)
identity-server-analytics-dashboard_1  |    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:141)
identity-server-analytics-dashboard_1  |    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645)
identity-server-analytics-dashboard_1  |    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
identity-server-analytics-dashboard_1  |    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
identity-server-analytics-dashboard_1  |    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
identity-server-analytics-dashboard_1  |    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886)
identity-server-analytics-dashboard_1  |    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
identity-server-analytics-dashboard_1  |    at java.lang.Thread.run(Thread.java:748)
identity-server-analytics-dashboard_1  | Caused by: javax.net.ssl.SSLException: Received fatal alert: bad_certificate
identity-server-analytics-dashboard_1  |    at sun.security.ssl.Alerts.getSSLException(Alerts.java:208)
identity-server-analytics-dashboard_1  |    at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1647)
identity-server-analytics-dashboard_1  |    at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1615)
identity-server-analytics-dashboard_1  |    at sun.security.ssl.SSLEngineImpl.recvAlert(SSLEngineImpl.java:1781)
identity-server-analytics-dashboard_1  |    at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:1070)
identity-server-analytics-dashboard_1  |    at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:896)
identity-server-analytics-dashboard_1  |    at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:766)
identity-server-analytics-dashboard_1  |    at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624)
identity-server-analytics-dashboard_1  |    at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:292)
identity-server-analytics-dashboard_1  |    at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1248)
identity-server-analytics-dashboard_1  |    at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1159)
identity-server-analytics-dashboard_1  |    at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1194)
identity-server-analytics-dashboard_1  |    at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489)
identity-server-analytics-dashboard_1  |    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428)
identity-server-analytics-dashboard_1  |    ... 16 more
identity-server-analytics-dashboard_1  | 

And analytics' dashboard is not detecting the Identity Server's login log. I'd like to use analytics properly and make it clear to detect login logs.

I'm using Ubuntu 18.04 LTS. How can I fix it?

Answer 1

Looking at the stack trace, seems your Analytics Server's public certificate is not imported in to the Identity Server's client-truststore.jks . (Requires a restart)

Export the public certificate of your Analytics server using the below command.

keytool -exportcert -keystore wso2carbon.jks -alias [alias] -file cert.cer

Import the extracted certificate to the Identity Server's client truststore using the below command.

keytool -importcert -file cert.cer -keystore client-truststore.jks -alias wso2-analytics

Restart the Identity Server.

PS

To find the alias value for the first command, you might have to list all the certificates of the Analytics Server's wso2carbon.jks using the below command and find the certificate denoted as the PrivateKeyEntry . Use the alias of that certificate.

keytool -list -v -keystore wso2carbon.jks