简体   繁体   中英

connection pooling with cassandra-driver for nodejs

i have wrote a small application to understand the concept connection pooling in cassandra. i am testing it using Apache bench tool. however the app sometimes shows the error and closes the connections in between. and then it throws error like host is down. when i give request for 1,00,000 queries with concurrency of 1000. it executes successfully but if i try to increase concurrency, then it ends up closing socket and eventually it shows host is down. so what my questions are is there a limit on concurrent requests that cassandra can serve? limit is approximateky 32K requests without waiting for response as i am using cassandra 3.0.

  1. does that means i can have that much concurrent request running?

  2. whats the reason behind getting this errors? for some inputs and not for all.

  3. how exactly connection pooling works in cassandra as we can only set the number of coreConnections per host and not the number of requests per connection?

here is the test file i made.

var cassandra = require("cassandra-driver");
  var express    = require("express");
  var distance = cassandra.types.distance;


  var cassClient = new cassandra.Client({contactPoints: ['myhost'],pooling: {
        coreConnectionsPerHost: {
          [distance.local] : 2,
          [distance.remote] : 1
        } 
     }, keyspace: 'MY_DB'});

  var app = express();
    cassClient.connect(function( err ){
      if( err ){
        console.log("Error: Connection to cassandra server."+err);
      } else {
        console.log("Success !!!");
      }
    });

  var i =0;
  app.get("/",function(req,res){
  cassClient.execute('SELECT * from mytable LIMIT 2', function(err, rows) {
  //cassClient.shutdown();
    if (!err){   
    //  console.log('The solution is: ' +i);
      return res.send();
      }
    else
      {

        i++;
        console.log(err.message +"  "+i);
        return res.send();
      }
    });
   // cassClient.nwd();
  });

  app.listen(3001);

the errors i am getting are

  • All host(s) tried for query failed. First host tried, myhost:9042: Host considered as DOWN. See innerErrors.
  • All host(s) tried for query failed. First host tried, myhost:9042: Error: Socket was closed. See innerErrors.
  • All host(s) tried for query failed. First host tried, myhost:9042: OperationTimedOutError: The host myhost:9042 did not reply before timeout 12000 ms. See innerErrors.

so why is it happening?

The following error:

All host(s) tried for query failed. First host tried, 
myhost:9042: OperationTimedOutError: The host myhost:9042 did not reply before 
timeout 12000 ms. See innerErrors.

Refers to client timeouts: when a request is issued but it's taking too long to get the response from the server (see socketOptions.readTimeout ).

When too many requests have simultaneously timed out, the connection is marked as defunct (see socketOptions.defunctReadTimeoutThreshold ) and removed from the pool. If the pool is empty, the node is marked as DOWN.

You can enable logging at level INFO or above to see what's happening under the hood.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM