使用 scala 向下播放框架 2.7

Question

我有一個在 play 2.7 上運行的 play scala 應用程序。 這用作我們前端的中間件，它具有 rest 端點。

現在我在雲上運行兩個不同的實例並使用 nginx 並綁定這兩個服務器並使用循環進行負載平衡。

現在我遇到一個問題，即服務器經常宕機，即每天 3 次，有趣的是兩台服務器同時宕機。 當我查看它時，兩台服務器上都出現了 memory 異常。 我試圖打印出 memory 的 javaheapdump 但沒有轉儲。 我仍在分析線程轉儲，以找出導致我的服務器宕機的實際原因，但讓我感到困惑的是兩台服務器同時宕機的原因。

在線程轉儲中，我看到有 7707 個線程在休眠 state。 這里是

"Connection evictor" #146 daemon prio=5 os_prio=0 cpu=2.33ms elapsed=1822.02s tid=0x00007f8a840c4800 nid=0x194 waiting on condition  [0x00007f8a58a5e000]
  java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep(java.base@11/Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(java.base@11/Thread.java:834)

這是我在服務器宕機時看到的

[35966.967s][warning][os,thread] Failed to start thread - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 0k, detached.
Uncaught error from thread [application-akka.actor.default-dispatcher-1398Uncaught error from thread [application-akka.actor.default-dispatcher-1395]: ]: unable to create native thread: possibly out of memory or process/resource limits reachedunable to create native thread: possibly out of memory or process/resource limits reached, shutting down JVM since 'akka.jvm-exit-on-fatal-error' is enabled for, shutting down JVM since 'akka.jvm-exit-on-fatal-error' is enabled for ActorSystem[ ActorSystem[applicationapplication]
]
java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
  at java.base/java.lang.Thread.start0(Native Method)
  at java.base/java.lang.Thread.start(Thread.java:803)
  at org.apache.http.impl.client.IdleConnectionEvictor.start(IdleConnectionEvictor.java:96)
  at org.apache.http.impl.client.HttpClientBuilder.build(HttpClientBuilder.java:1219)
  at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:287)
  at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:298)
  at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:236)
  at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:223)
  at org.apache.solr.client.solrj.impl.HttpSolrClient.<init>(HttpSolrClient.java:198)
  at org.apache.solr.client.solrj.impl.HttpSolrClient$Builder.build(HttpSolrClient.java:934)
  at com.github.takezoe.solr.scala.SolrClient$.$anonfun$$lessinit$greater$default$2$1(SolrClient.scala:11)
  at com.github.takezoe.solr.scala.SolrClient.<init>(SolrClient.scala:14)
  at service.tvt.solr.SolrPolygonService.getSuburbBoundary(SolrPolygonService.scala:212)
  at service.tvt.search.OrbigoSearchService.mapfeeder(OrbigoSearchService.scala:54)
  at service.bto.business_categories.MeedssCountService.$anonfun$suburbMeedssCount$2(MeedssCountService.scala:81)
  at scala.collection.immutable.List.map(List.scala:287)
  at service.bto.business_categories.MeedssCountService.suburbMeedssCount(MeedssCountService.scala:80)
  at controllers.bto.industry_categories.meedss.MeedssController.$anonfun$suburbMeedssCount$1(MeedssController.scala:38)
  at play.api.mvc.ActionBuilder.$anonfun$apply$11(Action.scala:368)
  at scala.Function1.$anonfun$andThen$1(Function1.scala:52)
  at play.api.mvc.ActionBuilderImpl.invokeBlock(Action.scala:489)
  at play.api.mvc.ActionBuilderImpl.invokeBlock(Action.scala:487)
  at play.api.mvc.ActionBuilder$$anon$9.invokeBlock(Action.scala:336)
  at play.api.mvc.ActionBuilder$$anon$9.invokeBlock(Action.scala:331)
  at play.api.mvc.ActionBuilder$$anon$10.apply(Action.scala:426)
  at play.api.mvc.Action.$anonfun$apply$2(Action.scala:98)
  at play.api.libs.streams.StrictAccumulator.$anonfun$mapFuture$4(Accumulator.scala:184)
  at scala.util.Try$.apply(Try.scala:209)
  at play.api.libs.streams.StrictAccumulator.$anonfun$mapFuture$3(Accumulator.scala:184)
  at akka.stream.impl.Transform.apply(TraversalBuilder.scala:159)
  at akka.stream.impl.PhasedFusingActorMaterializer.materialize(PhasedFusingActorMaterializer.scala:515)
  at akka.stream.impl.PhasedFusingActorMaterializer.materialize(PhasedFusingActorMaterializer.scala:450)
  at akka.stream.impl.PhasedFusingActorMaterializer.materialize(PhasedFusingActorMaterializer.scala:443)
  at akka.stream.scaladsl.RunnableGraph.run(Flow.scala:629)
  at play.api.libs.streams.Accumulator$.$anonfun$futureToSink$2(Accumulator.scala:262)
  at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:303)
  at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:37)
  at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
  at play.api.libs.streams.Execution$trampoline$.execute(Execution.scala:72)
  at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:68)
  at scala.concurrent.impl.Promise$DefaultPromise.dispatchOrAddCallback(Promise.scala:312)
  at scala.concurrent.impl.Promise$DefaultPromise.onComplete(Promise.scala:303)
  at scala.concurrent.impl.Promise.transformWith(Promise.scala:36)
  at scala.concurrent.impl.Promise.transformWith$(Promise.scala:34)
  at scala.concurrent.impl.Promise$DefaultPromise.transformWith(Promise.scala:183)
  at scala.concurrent.Future.flatMap(Future.scala:302)
  at scala.concurrent.Future.flatMap$(Future.scala:302)
  at scala.concurrent.impl.Promise$DefaultPromise.flatMap(Promise.scala:183)
  at play.api.libs.streams.Accumulator$.$anonfun$futureToSink$1(Accumulator.scala:261)
  at akka.stream.impl.Transform.apply(TraversalBuilder.scala:159)
  at akka.stream.impl.PhasedFusingActorMaterializer.materialize(PhasedFusingActorMaterializer.scala:515)
  at akka.stream.impl.PhasedFusingActorMaterializer.materialize(PhasedFusingActorMaterializer.scala:450)
  at akka.stream.impl.PhasedFusingActorMaterializer.materialize(PhasedFusingActorMaterializer.scala:443)
  at akka.stream.scaladsl.RunnableGraph.run(Flow.scala:629)
  at play.api.libs.streams.SinkAccumulator.run(Accumulator.scala:144)
  at play.api.libs.streams.SinkAccumulator.run(Accumulator.scala:148)
  at play.core.server.AkkaHttpServer.$anonfun$runAction$4(AkkaHttpServer.scala:441)
  at akka.http.scaladsl.util.FastFuture$.strictTransform$1(FastFuture.scala:41)
  at akka.http.scaladsl.util.FastFuture$.$anonfun$transformWith$3(FastFuture.scala:51)
  at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
  at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
  at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:92)
  at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
  at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:81)
  at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:92)
  at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:41)
  at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:49)
  at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

任何快速指示都會非常有幫助

Answer 1

Levi Ramsey 是對的，因為我們正在使用 TakeZoe lib。 我們為每個新請求創建客戶端，而不是關閉它。 最后，我們創建了一個活動連接有限的連接池，並且它工作了。