如何獲取Spark Java應用程序的自動定義端口？

Question

在Java Spark的API文檔（不是Apache spark）中，您可以指定端口0以使其自動選擇端口。 大！

但是，我無法弄清楚如何在服務器啟動后獲取該端口。 我可以在日志中看到它：

15:41:12.459 [Thread-2] INFO  spark.webserver.JettySparkServer - >> Listening on 0.0.0.0:63134

但我需要能夠以編程方式實現它，以便我的集成測試能夠每次都可靠地運行。

那么我該如何獲得該端口？

Answer 1

我找不到在API中獲取此信息的方法，因此我在他們的github上提交了一個問題 。

我能夠通過一堆丑陋的反射得到它：

/**
 * Meant to be called from a different thread, once the spark app is running
 * This is probably only going to be used during the integration testing process, not ever in prod!
 *
 * @return the port it's running on
 */
public static int awaitRunningPort() throws Exception {
    awaitInitialization();
    //I have to get the port via reflection, which is fugly, but the API doesn't exist :(
    //Since we'll only use this in testing, it's not going to kill us
    Object instance = getInstance();
    Class theClass = instance.getClass();
    Field serverField = theClass.getDeclaredField("server");
    serverField.setAccessible(true);
    Object oneLevelDeepServer = serverField.get(instance);

    Class jettyServerClass = oneLevelDeepServer.getClass();
    Field jettyServerField = jettyServerClass.getDeclaredField("server");
    jettyServerField.setAccessible(true);
    //Have to pull in the jetty server stuff to do this mess
    Server jettyServer = (Server)jettyServerField.get(oneLevelDeepServer);

    int acquiredPort = ((ServerConnector)jettyServer.getConnectors()[0]).getLocalPort();

    log.debug("Acquired port: {}", acquiredPort);
    return acquiredPort;
}

這在我們的集成測試中對我很有用，但我沒有使用https，它通過反射抓取受保護的字段確實達到了API的兩個層次。 我找不到任何其他方法來做到這一點。 很高興被證明是錯誤的。

Answer 2

這將適用於Spark 2.6.0：

public static int start (String keystoreFile, String keystorePw)
{
    secure(keystoreFile, keystorePw, null, null);
    port(0);

    staticFiles.location("/public");

    get(Path.CLOCK, ClockController.time);
    get(Path.CALENDAR, CalendarController.date);

    // This is the important line. It must be *after* creating the routes and *before* the call to port()
    awaitInitialization();

    return port();
}

如果沒有調用awaitInitialization（），port（）將返回0。