[英]datastax driver connection = apache ignite and cassandra(you may want to increase the driver number of per-host connections)
Components: apache ignite + apache cassandra. 成分:apache点燃+ apache cassandra。 Use defaut datastax driver.
使用defaut datastax驱动程序。 After doing several operation(about 3-5 billions entities put to cache) we get a situation when datastax driver always reconnects to cassandra from ignite.
经过几次操作(大约有3-5亿个实体被缓存),我们发现datastax驱动程序总是从点燃重新连接到cassandra。
2017-02-16 13:29:21.287 INFO 160487 --- [ sys-#441%null%] m.r.t.d.c.m.p.c.c.p.RetryPolicyImpl : init cluster
2017-02-16 13:29:21.288 INFO 160487 --- [ sys-#441%null%] com.datastax.driver.core.Cluster : New Cassandra host <our host> added
2017-02-16 13:29:21.307 INFO 160487 --- [ sys-#441%null%] m.r.t.d.c.m.p.c.c.p.RetryPolicyImpl : close cluster
2017-02-16 13:29:23.516 INFO 160487 --- [ sys-#441%null%] com.datastax.driver.core.ClockFactory : Using native clock to generate timestamps.
2017-02-16 13:29:23.537 INFO 160487 --- [ sys-#441%null%] c.d.d.c.p.DCAwareRoundRobinPolicy : Using data-center name 'datacenter1' for DCAw
And this process is endless and it can be interrupted by server restarted. 并且此过程是无止境的,可以通过重新启动服务器来中断。
Infrastructure : 1 server ignite - ~Xmx30g and 8 cores. 基础架构:点燃1个服务器-〜Xmx30g和8个核心。 25 clients ignites ~Xmx1g and 8 cores.
25个客户端点燃〜Xmx1g和8个内核。 1 node cassandra.
1个节点cassandra。 Batch size(entities which will be put to cache and then cassandra) is about 1-2M.
批处理大小(将被缓存然后放入cassandra的实体)约为1-2M。
Config datasource => 配置数据源=>
public DataSource dataSource() {
DataSource dataSource = new DataSource();
dataSource.setUser(login);
dataSource.setPassword(pass);
dataSource.setPort(port);
dataSource.setContactPoints(host);
dataSource.setRetryPolicy(retryPolicy);
dataSource.setFetchSize(10_000);
dataSource.setReconnectionPolicy(new ConstantReconnectionPolicy(1000));
dataSource.setLoadBalancingPolicy(DCAwareRoundRobinPolicy.builder().withUsedHostsPerRemoteDc(0).build());
dataSource.setSocketOptions(new SocketOptions().setReadTimeoutMillis(100_000).setConnectTimeoutMillis(100_00));
return dataSource;
}
config cache => 配置缓存=>
CacheConfiguration<KeyIgnite, Long> cfg = new CacheConfiguration<>();
cfg
.setName(area)
.setRebalanceMode(CacheRebalanceMode.SYNC)
.setStartSize(1_000_000)
.setAtomicityMode(CacheAtomicityMode.ATOMIC)
.setIndexedTypes(KeyIgnite.class, Long.class)
.setCacheMode(CacheMode.PARTITIONED)
.setBackups(0);
if (!clientMode) {
CassandraCacheStoreFactory<KeyIgnite, Long> csFactory = new CassandraCacheStoreFactory<>();
csFactory.setDataSource(ds);
csFactory.setPersistenceSettings(kv);
// CassandraCacheStoreFactoryDwh<KeyIgnite, Long> csFactory = new CassandraCacheStoreFactoryDwh<>(ds, kv,params);
cfg
.setCacheStoreFactory(csFactory)
.setReadThrough(true)
.setWriteThrough(true);
}
cfg.setExpiryPolicyFactory(TouchedExpiryPolicy.factoryOf(new Duration(TimeUnit.DAYS, 5)));
return cfg;
ignite config =>: 点燃配置=>:
TcpDiscoveryMulticastIpFinder finder = new TcpDiscoveryMulticastIpFinder();
finder.setAddresses(adresses);
return Ignition.start(
new IgniteConfiguration()
.setClientMode(clientMode)
.setDiscoverySpi(new TcpDiscoverySpi().setIpFinder(finder).setNetworkTimeout(10000))
.setFailureDetectionTimeout(50000)
.setPeerClassLoadingEnabled(false)
.setLoadBalancingSpi(new RoundRobinLoadBalancingSpi())
);
When we have done several iteration of putting items to cache we have gotten this case. 当我们完成了几次将项目放入缓存的迭代时,就得到了这种情况。
after including debug level i have gotten this record: 包括调试级别后,我得到了以下记录:
2017-02-17 17:48:41.570 DEBUG 24816 --- [ sys-#184%null%] com.datastax.driver.core.RequestHandler : [1071994376-1] Error querying ds-inmemorydb-02t/10.216.28.34:9042 : com.datastax.driver.core.exceptions.DriverException: Timeout while trying to acquire available connection (you may want to increase the driver number of per-host connections)
驱动程序的超时增加有助于
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.