I'm using HTTPBuilder to build a client for the REST API of a NoSQL database. It works fine except when processing a large data set. After a large number of calls the client throws this exception:
java.net.SocketException: No buffer space available (maximum connections reached?): connect
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:640)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:479)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:446)
at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:417)
at groovyx.net.http.HTTPBuilder.request(HTTPBuilder.java:366)
at groovyx.net.http.HTTPBuilder$request.call(Unknown Source)
at scripts.groovy.prototypes.NeoClient.getRelationship(NeoClient.groovy:70)
at com.gsgenetics.genie.trait.Node$Edge.getTargetNode(Node.java:108)
at com.gsgenetics.genie.trait.Node$Edge.compareTo(Node.java:145)
at com.gsgenetics.genie.trait.Node$Edge.compareTo(Node.java:67)
at com.gsgenetics.genie.trait.Node.getEdges(Node.java:32)
at com.gsgenetics.genie.trait.Node.iterator(Node.java:384)
at com.gsgenetics.genie.GenomeStructuredGraph.add(GenomeStructuredGraph.java:131)
at com.gsgenetics.genie.GenomeStructuredGraph.add(GenomeStructuredGraph.java:197)
at com.gsgenetics.genie.GenomeStructuredGraph$add.call(Unknown Source)
at scripts.groovy.prototypes.read-ccds$_run_closure3.doCall(read-ccds.groovy:53)
at scripts.groovy.prototypes.read-ccds.run(read-ccds.groovy:50)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
It looks like some resources are not being released but I can' figure out what exactly and how to get them to release. I am running this on Windows 7 but I get a similar error when running it on Linux.
Here's the code where I am instantiating and using HTTPBuilder:
import groovyx.net.http.*
import net.sf.json.JSONArray
import net.sf.json.JSONObject
import static groovyx.net.http.ContentType.*
import static groovyx.net.http.Method.*
class NeoClient {
static final URL = 'http://some IP address'
// static def http = new HTTPBuilder( URL )
static net.sf.json.JSONObject query(statement, params=null, success=null, error=null) {
def http = new HTTPBuilder( URL )
net.sf.json.JSONObject returnJson = []
http.request( POST, JSON ) {
uri.path = '/db/data/cypher/'
headers.'X-Stream' = 'true'
requestContentType = JSON
body = [ query : statement , params : params ?: [:] ]
response.success = { resp, json ->
if (success) success(json)
else {
println "Status ${resp.statusLine} Columns ${json.columns}\nData: ${json.data}"
}
returnJson = json
}
response.failure = { resp, message ->
def result=[status:resp.statusLine.statusCode,statusText:resp.statusLine.reasonPhrase]
result.headers = resp.headers.collect { h -> [ (h.name) : h.value ] }
result.message = message
if (error) {
error(result)
} else {
println "Status: ${result.status} : ${result.statusText} "
println "Headers: ${result.headers}"
println "Message: ${result.message}"
}
throw new Exception("Neo Client Exception");
}
}
}
}
Note that I tried having a single instance of HTTPBuilder in my static class NeoClient, and also creating a new instance of HTTPBuilder for each call to "query", and I get the same exception in either case.
Any ideas would be greatly appreciated.
Thanks
You should use the HTTPBuilder.shutdown method for releasing any system resources held by this instance.
In addition, creating a new HTTPBuilder
for each request is not the best approach in terms of performance and resource management. A better approach can be having a single instance of HttpBuilder
which handles multiple requests. To do so, you will need to override the default behavior of HttpBuilder
to use a thread safe connection manager:
private static class CustomHttpBuilder extends HttpBuilder {
protected AbstractHttpClient createClient( HttpParams params ) {
def connManager = new MultiThreadedHttpConnectionManager()
def connManagerParams = new HttpConnectionManagerParams()
connManagerParams.maxTotalConnections = 50 // default is 20
connManagerParams.defaultMaxConnectionsPerHost = 50 // default is 2
connManager.params = connManagerParams
new HttpClient(connManager)
}
}
I got this code example from the Grails users forum . This will allow you to use a single instance of the CustomHttpBuilder
for multiple requests.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.