Redis、Jedis 連接池優化和監控工具

Question

我是 Redis 的新手，Jedis。 今天，我在我的日志中看到一個錯誤

Caused by: redis.clients.jedis.exceptions.JedisExhaustedPoolException: redis.clients.jedis.exceptions.JedisExhaustedPoolException: Could not get a resource since the pool is exhausted
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_191]
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_191]
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_191]
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_191]
        at java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:593) ~[?:1.8.0_191]
        at java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:677) ~[?:1.8.0_191]
        at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:735) ~[?:1.8.0_191]
        at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:160) ~[?:1.8.0_191]
        at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:174) ~[?:1.8.0_191]
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233) ~[?:1.8.0_191]
        at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) ~[?:1.8.0_191]
        at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:583) ~[?:1.8.0_191]
        at com.nokia.snmpapp.VariableDataUtils.processSnmpDataByOP(VariableDataUtils.java:372) ~[snmpapp.jar:?]
       at com.nokia.snmpapp.consumerservice.DocsisQos3MibData.processDocsQos3ParamSetTableObjectEvent(DocsisQos3MibData.java:172) ~[snmpapp.jar:?]
        at com.nokia.snmpapp.KafkaConsumer.onSnmpappDocsQos3ParamSetTableObjectEvent(KafkaConsumer.java:315) ~[snmpapp.jar:?]
        ... 16 more 

Caused by: redis.clients.jedis.exceptions.JedisExhaustedPoolException: Could not get a resource since the pool is exhausted
            at redis.clients.jedis.util.Pool.getResource(Pool.java:53) ~[snmpapp.jar:?]
            at redis.clients.jedis.JedisPool.getResource(JedisPool.java:234) ~[snmpapp.jar:?]
            at com.nokia.snmpapp.RedisUtils.add2list(RedisUtils.java:27) ~[snmpapp.jar:?]
            at com.nokia.snmpapp.VariableDataUtils.lambda$processSnmpDataByOP$5(VariableDataUtils.java:379) ~[snmpapp.jar:?]
            at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) ~[?:1.8.0_191]
            at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382) ~[?:1.8.0_191]
            at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) ~[?:1.8.0_191]
            at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291) ~[?:1.8.0_191]
            at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731) ~[?:1.8.0_191]
            at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) ~[?:1.8.0_191]
            at java.util.concurrent.ForkJoinPool$WorkQueue.execLocalTasks(ForkJoinPool.java:1040) ~[?:1.8.0_191]
            at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1058) ~[?:1.8.0_191]
            at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) ~[?:1.8.0_191]
            at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) ~[?:1.8.0_191]  

Caused by: java.util.NoSuchElementException: Timeout waiting for idle object               
           at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:439) ~[snmpapp.jar:?]
           at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:349) ~[snmpapp.jar:?]
           at redis.clients.jedis.util.Pool.getResource(Pool.java:50) ~[snmpapp.jar:?]
           at redis.clients.jedis.JedisPool.getResource(JedisPool.java:234) ~[snmpapp.jar:?]
           at com.nokia.snmpapp.RedisUtils.add2list(RedisUtils.java:27) ~[snmpapp.jar:?]
           at com.nokia.snmpapp.VariableDataUtils.lambda$processSnmpDataByOP$5(VariableDataUtils.java:379) ~[snmpapp.jar:?]
           at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) ~[?:1.8.0_191]
           at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382) ~[?:1.8.0_191]
           at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) ~[?:1.8.0_191]
           at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291) ~[?:1.8.0_191]
           at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731) ~[?:1.8.0_191]
           at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) ~[?:1.8.0_191]
           at java.util.concurrent.ForkJoinPool$WorkQueue.execLocalTasks(ForkJoinPool.java:1040) ~[?:1.8.0_191]
           at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1058) ~[?:1.8.0_191]
           at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) ~[?:1.8.0_191]
           at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) ~[?:1.8.0_191]

我目前的配置是

  maxTotal: 10
  maxIdle: 5
  minIdle: 1
  maxWaitMillis: 2000

我知道我可以增加連接池大小來解決錯誤消息。 但是，在為我的 env 確定一個好的數字之前，我想更多地了解資源使用情況。 我一直在用谷歌搜索這個，但我找不到太多關於監控工具或如何監控性能和池使用情況的討論。 這是否意味着沒有工具可以監控？ 如果無法監控連接池使用情況，我該怎么做才能更好地了解在 Redis 連接上優化什么？

提前致謝

Answer 1

使用 JMX MBean 監控 Jedis 池

Jedis Pool 基於 Apache Generic-Pool API，因此您可以使用 JMX 來監控連接池的狀態。

例如，在啟用 JMX 的情況下啟動您的應用程序：

-Dcom.sun.management.jmxremote.port=9999 
-Dcom.sun.management.jmxremote.authenticate=false 
-Dcom.sun.management.jmxremote.ssl=false

然后連接 JConsole 或任何其他 JMX 兼容工具：

1. 連接到您的應用程序/流程
2. 轉到 MBean
3. 轉到org.apache.commons.pool2.GenericObjectPool.pool MBeans

您將能夠監控您的應用程序/JedisPool 發生的情況。

您有許多符合 JMX 的監控工具，允許您根據閾值發出警報

---

大小調整和配置

池的配置實際上是特定於應用程序的，要配置maxTotal屬性，即您必須查看的最大連接數：

• 您的服務需要多少並發連接
• 每次調用執行您發送的命令需要多長時間

並且您需要確保您的系統不會超載（客戶端和服務器端。）

---

JedisPool：確保釋放連接；)

此外，這不是關於監控，而是關於最佳實踐：

• 您的應用程序是否在每次使用后釋放與池的連接？

/// Jedis implements Closeable. Hence, the jedis instance will be auto-closed after the last statement.
try (Jedis jedis = pool.getResource()) {

   // use your jedis
  jedis.set("foo", "bar");

} // the resource will be released back to the pool

如果不將 try 與資源一起使用，則必須關閉連接（將其放回池中）

Jedis jedis = null;
try {
  jedis = pool.getResource();
  // work with Redis
  jedis.set("foo", "bar");
} finally {
  // you must close the connection to put it back to the pool
  if (jedis != null) {
    jedis.close();
  }
}

同樣，當您關閉應用程序時：

 pool.close()