java.lang.UnsatisfiedLinkError 由“打開的文件太多”錯誤導致

Question

我正在嘗試診斷 JNA 嘗試加載“c”庫時發生的大型 3rd 方應用程序 (Midonet) 中的問題。 日志條目（帶有堆棧跟蹤）如下。

當我嘗試在具有 Linux 5.3 kernel 的新機器上啟動 midonet 時，問題就出現了。 它正在使用較舊的 kernel (4.15)。 打開的文件限制是 4096，所以我假設它是庫加載（以某種方式）打開大量文件。

現在 Midonet 使用的是相當舊的 JNA (4.2.2) 版本，所以我碰上了最后一個 4.x 版本，看看它是否有幫助。 我可以嘗試的下一件事是在平台本身上重建 JNA 4.x。

我無法在 JNA 或 Midonet 問題跟蹤器中找到任何相關問題。 或與 Google 搜索相關的任何內容

我想知道之前是否有其他人看到過此錯誤（使用 Midolnet 或任何其他 Java 應用程序）。 其他有關如何調試的建議將不勝感激。

這是堆棧跟蹤：

2020.05.22 16:49:40.212 ERROR [agent-services-pool-1] jna -  Native method calls are not available
java.lang.UnsatisfiedLinkError: Failed to create temporary file for /com/sun/jna/linux-x86-64/libjnidispatch.so library: Too many open files
    at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:962) ~[midolman.jar:5.4.6-Nectar-SNAPSHOT]
    at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:922) ~[midolman.jar:5.4.6-Nectar-SNAPSHOT]
    at com.sun.jna.Native.<clinit>(Native.java:190) ~[midolman.jar:5.4.6-Nectar-SNAPSHOT]
    at org.midonet.jna.CLibrary.<clinit>(CLibrary.java:45) ~[midolman.jar:5.4.6-Nectar-SNAPSHOT]
    at org.midonet.netlink.UnixDomainChannel.<init>(UnixDomainChannel.java:61) [midolman.jar:5.4.6-Nectar-SNAPSHOT]
    at org.midonet.netlink.UnixDomainChannelImpl.<init>(UnixDomainChannelImpl.java:32) [midolman.jar:5.4.6-Nectar-SNAPSHOT]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) [na:1.8.0_252]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) [na:1.8.0_252]
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [na:1.8.0_252]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423) [na:1.8.0_252]
    at org.midonet.netlink.NetlinkSelectorProvider.makeInstanceOf(NetlinkSelectorProvider.java:143) [midolman.jar:5.4.6-Nectar-SNAPSHOT]
    at org.midonet.netlink.NetlinkSelectorProvider.openUnixDomainSocketChannel(NetlinkSelectorProvider.java:118) [midolman.jar:5.4.6-Nectar-SNAPSHOT]
    at org.midonet.services.rest_api.hacks.UnixDomainServerSocketChannel$.open(UnixDomainServerSocketChannel.scala:38) [midolman.jar:5.4.6-Nectar-SNAPSHOT]
    at org.midonet.services.rest_api.hacks.UnixDomainServerSocketChannel.open(UnixDomainServerSocketChannel.scala) [midolman.jar:5.4.6-Nectar-SNAPSHOT]
    at org.midonet.services.rest_api.hacks.UnixDomainServerConnector.open(UnixDomainServerConnector.java:341) [midolman.jar:5.4.6-Nectar-SNAPSHOT]
    at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80) [midolman.jar:5.4.6-Nectar-SNAPSHOT]
    at org.midonet.services.rest_api.hacks.UnixDomainServerConnector.doStart(UnixDomainServerConnector.java:258) [midolman.jar:5.4.6-Nectar-SNAPSHOT]
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) [midolman.jar:5.4.6-Nectar-SNAPSHOT]
    at org.eclipse.jetty.server.Server.doStart(Server.java:384) [midolman.jar:5.4.6-Nectar-SNAPSHOT]
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) [midolman.jar:5.4.6-Nectar-SNAPSHOT]
    at org.midonet.services.rest_api.BindingApiService.doStart(BindingApiService.scala:113) [midolman.jar:5.4.6-Nectar-SNAPSHOT]
    at com.google.common.util.concurrent.AbstractService.startAsync(AbstractService.java:203) [midolman.jar:5.4.6-Nectar-SNAPSHOT]
    at org.midonet.minion.Daemon$$anonfun$org$midonet$minion$Daemon$$startMinion$1.apply(Daemon.scala:87) [midolman.jar:5.4.6-Nectar-SNAPSHOT]
    at org.midonet.minion.Daemon$$anonfun$org$midonet$minion$Daemon$$startMinion$1.apply(Daemon.scala:83) [midolman.jar:5.4.6-Nectar-SNAPSHOT]
    at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24) [midolman.jar:5.4.6-Nectar-SNAPSHOT]
    at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24) [midolman.jar:5.4.6-Nectar-SNAPSHOT]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_252]
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_252]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_252]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.8.0_252]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_252]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_252]
    at java.lang.Thread.run(Thread.java:748) [na:1.8.0_252]

觸發代碼的 Midonet 代碼是：

import com.sun.jna.Native;
import com.sun.jna.Platform;

...

static {
    try {
        Native.register(Platform.C_LIBRARY_NAME);
    } catch (NoClassDefFoundError | UnsatisfiedLinkError | NoSuchMethodError e) {
        log.error("Native method calls are not available", e);
        System.exit(-1);
    }
}

Answer 1

事實證明，問題根本不是 JNA 的錯。

事實證明，midolnet minion 進程在 Linux 5.3 kernel 上運行時確實需要 > 4096 個打開的文件。 在minion-start腳本中設置ulimit -n 5000可以修復它。