Cassandra datastax OperationTimedOutException

Question

I'm using a 3 nodes cassandra 3.0.14 deployed on 3 differents VM. I have a lots of data (billions) and I would like to make quick search among my Cassandra architecture.

I've made a lots of research on Cassandra but I'm still facing some issues that I cannot understand:

1- When I am using cqlsh I can make a query that analyzes all my database

SELECT DISTINCT val_1 FROM myTable; is working.

However I cannot make the same request using my java code and datastax driver. My script return:

Caused by: com.datastax.driver.core.exceptions.OperationTimedOutException: [/XX.XX.XX.XX:9042] Timed out waiting for server response

2- Some request are working using cqlsh but making a more specific request will lead to a request timeout:

OperationTimedOut: errors={'127.0.0.1': 'Client request timeout. See Session.execute[_async](timeout)'}, last_host=127.0.0.1

For example if I'm making this request:

SELECT val_1 FROM myTable where time>'2018-09-16 09:00:00'; will work SELECT val_1 FROM myTable where time>'2018-09-16 09:00:00' and time<'2018-09-17 09:00:00'; will lead to time out

I changed my request_timeout_in_ms to 60s but I know it is not a good practice. I also increase my read_request_timeout_in_ms and range_request_timeout_in_ms but I still have the previous issues.

Would anyone have the same problems ?

-Nicolas

Answer 1

Try to adjust the client timeout in Java code, as follows:

//configure socket options
SocketOptions options = new SocketOptions();
options.setConnectTimeoutMillis(30000);
options.setReadTimeoutMillis(300000);
options.setTcpNoDelay(true);

//spin up a fresh connection (using the SocketOptions set up above)
cluster = Cluster.builder().addContactPoint(Configuration.getCassandraHost()).withPort(Configuration.getCassandraPort())
            .withCredentials(Configuration.getCassandraUser(), Configuration.getCassandraPass()).withSocketOptions(options).build();

Answer 2

This happens because you're using Cassandra incorrect way. The range operations, distinct, etc. works best only if you have the partition key specified in your query. Otherwise, Cassandra will need to scan whole cluster trying to find the data that you need, and this will lead to timeout even on the medium-sized database. Don't use the ALLOW FILTERING to enforce execution of the queries.

In Cassandra, database structure is modeled around the queries that you want to execute. I recommend to take DS201 & DS220 courses from the DataStax Academy .