Apache ODE on OpenShift calling remote web-service timeout

Question

I have the following configuration:

Tomcat 7 gear running on OpenShift Online. I deployed Apache ODE as a webapp and tested a simple BPEL workflow (Echo test) that worked perfectly. I have 2 more gears on OpenShift both running WildFly8 servers and some web-services. I created a workflow for ODE that calles these services. When I test the workflow on my local PC running Tomcat 7 and Apache ODE it works just fine. However, on OpenShift I have the following issue:

When ODE tries to call a remote web-service it tells its Axis libraries to create a socket. Since binding sockets to "localhost" on OpenShift is not allowed, I get an exception. I modified the org.apache.commons.httpclient.HttpConnection.open() method to bind the socket to the local OpenShift IP of my gear instead of localhost and got rid of the bind exception. However, the remote web-service doesn't seem to respond to my request:

07:11:50,505 ERROR [ExternalService] Error sending message (mex={PartnerRoleMex#hqejbhcnphr9mgvfg5xbh1 [PID {org.neo}btest-2] calling org.apache.ode.bpel.epr.WSAEndpoint@1f51919.getClientData(...) Status ASYNC}): The host did not accept the connection within timeout of 60000 ms
org.apache.axis2.AxisFault: The host did not accept the connection within timeout of 60000 ms
    at org.apache.axis2.AxisFault.makeFault(AxisFault.java:430)
    at org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:203)
    at org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:76)
    at org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:400)
    at org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:225)
    at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:438)
    at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:402)
    at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:229)
    at org.apache.axis2.client.OperationClient.execute(OperationClient.java:165)
    at org.apache.ode.axis2.SoapExternalService$1$1.call(SoapExternalService.java:206)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.commons.httpclient.ConnectTimeoutException: The host did not accept the connection within timeout of 60000 ms
    at org.apache.commons.httpclient.protocol.ReflectionSocketFactory.createSocket(ReflectionSocketFactory.java:155)
    at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:125)
    at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:715)
    at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361)
    at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
    at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
    at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
    at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:346)
    at org.apache.axis2.transport.http.AbstractHTTPSender.executeMethod(AbstractHTTPSender.java:557)
    at org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:199)
    ... 12 more
Caused by: java.net.SocketTimeoutException: connect timed out
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:579)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.commons.httpclient.protocol.ReflectionSocketFactory.createSocket(ReflectionSocketFactory.java:140)
    ... 21 more

This is the article that gave me this idea: https://www.openshift.com/forums/openshift/commons-httpclient-permission-denied

Again, when testing the exact same workflow on my PC with the exact same web-services it works just fine, so the WildFly8 servers and web-services are not the issue.

I am grateful for any tips, I've been at this for almost a week now...

Answer 1

I managed to fix it by creating a DIY cartridge and installing the newest version of Tomcat 7 onto it. I deployed ODE and my workflow, got the same binding error as before, modified the apache-commons library that Axis was using and now it's sailing smoothly.

I don't know why it didn't work with the Tomcat 7 cartridge I get from OpenShift as standard though.