使用 scala 向下播放框架 2.7

Question

我有一个在 play 2.7 上运行的 play scala 应用程序。 这用作我们前端的中间件，它具有 rest 端点。

现在我在云上运行两个不同的实例并使用 nginx 并绑定这两个服务器并使用循环进行负载平衡。

现在我遇到一个问题，即服务器经常宕机，即每天 3 次，有趣的是两台服务器同时宕机。 当我查看它时，两台服务器上都出现了 memory 异常。 我试图打印出 memory 的 javaheapdump 但没有转储。 我仍在分析线程转储，以找出导致我的服务器宕机的实际原因，但让我感到困惑的是两台服务器同时宕机的原因。

在线程转储中，我看到有 7707 个线程在休眠 state。 这里是

"Connection evictor" #146 daemon prio=5 os_prio=0 cpu=2.33ms elapsed=1822.02s tid=0x00007f8a840c4800 nid=0x194 waiting on condition  [0x00007f8a58a5e000]
  java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep(java.base@11/Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(java.base@11/Thread.java:834)

这是我在服务器宕机时看到的

[35966.967s][warning][os,thread] Failed to start thread - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 0k, detached.
Uncaught error from thread [application-akka.actor.default-dispatcher-1398Uncaught error from thread [application-akka.actor.default-dispatcher-1395]: ]: unable to create native thread: possibly out of memory or process/resource limits reachedunable to create native thread: possibly out of memory or process/resource limits reached, shutting down JVM since 'akka.jvm-exit-on-fatal-error' is enabled for, shutting down JVM since 'akka.jvm-exit-on-fatal-error' is enabled for ActorSystem[ ActorSystem[applicationapplication]
]
java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
  at java.base/java.lang.Thread.start0(Native Method)
  at java.base/java.lang.Thread.start(Thread.java:803)
  at org.apache.http.impl.client.IdleConnectionEvictor.start(IdleConnectionEvictor.java:96)
  at org.apache.http.impl.client.HttpClientBuilder.build(HttpClientBuilder.java:1219)
  at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:287)
  at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:298)
  at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:236)
  at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:223)
  at org.apache.solr.client.solrj.impl.HttpSolrClient.<init>(HttpSolrClient.java:198)
  at org.apache.solr.client.solrj.impl.HttpSolrClient$Builder.build(HttpSolrClient.java:934)
  at com.github.takezoe.solr.scala.SolrClient$.$anonfun$$lessinit$greater$default$2$1(SolrClient.scala:11)
  at com.github.takezoe.solr.scala.SolrClient.<init>(SolrClient.scala:14)
  at service.tvt.solr.SolrPolygonService.getSuburbBoundary(SolrPolygonService.scala:212)
  at service.tvt.search.OrbigoSearchService.mapfeeder(OrbigoSearchService.scala:54)
  at service.bto.business_categories.MeedssCountService.$anonfun$suburbMeedssCount$2(MeedssCountService.scala:81)
  at scala.collection.immutable.List.map(List.scala:287)
  at service.bto.business_categories.MeedssCountService.suburbMeedssCount(MeedssCountService.scala:80)
  at controllers.bto.industry_categories.meedss.MeedssController.$anonfun$suburbMeedssCount$1(MeedssController.scala:38)
  at play.api.mvc.ActionBuilder.$anonfun$apply$11(Action.scala:368)
  at scala.Function1.$anonfun$andThen$1(Function1.scala:52)
  at play.api.mvc.ActionBuilderImpl.invokeBlock(Action.scala:489)
  at play.api.mvc.ActionBuilderImpl.invokeBlock(Action.scala:487)
  at play.api.mvc.ActionBuilder$$anon$9.invokeBlock(Action.scala:336)
  at play.api.mvc.ActionBuilder$$anon$9.invokeBlock(Action.scala:331)
  at play.api.mvc.ActionBuilder$$anon$10.apply(Action.scala:426)
  at play.api.mvc.Action.$anonfun$apply$2(Action.scala:98)
  at play.api.libs.streams.StrictAccumulator.$anonfun$mapFuture$4(Accumulator.scala:184)
  at scala.util.Try$.apply(Try.scala:209)
  at play.api.libs.streams.StrictAccumulator.$anonfun$mapFuture$3(Accumulator.scala:184)
  at akka.stream.impl.Transform.apply(TraversalBuilder.scala:159)
  at akka.stream.impl.PhasedFusingActorMaterializer.materialize(PhasedFusingActorMaterializer.scala:515)
  at akka.stream.impl.PhasedFusingActorMaterializer.materialize(PhasedFusingActorMaterializer.scala:450)
  at akka.stream.impl.PhasedFusingActorMaterializer.materialize(PhasedFusingActorMaterializer.scala:443)
  at akka.stream.scaladsl.RunnableGraph.run(Flow.scala:629)
  at play.api.libs.streams.Accumulator$.$anonfun$futureToSink$2(Accumulator.scala:262)
  at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:303)
  at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:37)
  at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
  at play.api.libs.streams.Execution$trampoline$.execute(Execution.scala:72)
  at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:68)
  at scala.concurrent.impl.Promise$DefaultPromise.dispatchOrAddCallback(Promise.scala:312)
  at scala.concurrent.impl.Promise$DefaultPromise.onComplete(Promise.scala:303)
  at scala.concurrent.impl.Promise.transformWith(Promise.scala:36)
  at scala.concurrent.impl.Promise.transformWith$(Promise.scala:34)
  at scala.concurrent.impl.Promise$DefaultPromise.transformWith(Promise.scala:183)
  at scala.concurrent.Future.flatMap(Future.scala:302)
  at scala.concurrent.Future.flatMap$(Future.scala:302)
  at scala.concurrent.impl.Promise$DefaultPromise.flatMap(Promise.scala:183)
  at play.api.libs.streams.Accumulator$.$anonfun$futureToSink$1(Accumulator.scala:261)
  at akka.stream.impl.Transform.apply(TraversalBuilder.scala:159)
  at akka.stream.impl.PhasedFusingActorMaterializer.materialize(PhasedFusingActorMaterializer.scala:515)
  at akka.stream.impl.PhasedFusingActorMaterializer.materialize(PhasedFusingActorMaterializer.scala:450)
  at akka.stream.impl.PhasedFusingActorMaterializer.materialize(PhasedFusingActorMaterializer.scala:443)
  at akka.stream.scaladsl.RunnableGraph.run(Flow.scala:629)
  at play.api.libs.streams.SinkAccumulator.run(Accumulator.scala:144)
  at play.api.libs.streams.SinkAccumulator.run(Accumulator.scala:148)
  at play.core.server.AkkaHttpServer.$anonfun$runAction$4(AkkaHttpServer.scala:441)
  at akka.http.scaladsl.util.FastFuture$.strictTransform$1(FastFuture.scala:41)
  at akka.http.scaladsl.util.FastFuture$.$anonfun$transformWith$3(FastFuture.scala:51)
  at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
  at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
  at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:92)
  at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
  at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:81)
  at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:92)
  at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:41)
  at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:49)
  at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

任何快速指示都会非常有帮助

Answer 1

Levi Ramsey 是对的，因为我们正在使用 TakeZoe lib。 我们为每个新请求创建客户端，而不是关闭它。 最后，我们创建了一个活动连接有限的连接池，并且它工作了。