應用程序部署在兩個節點上時如何僅讀取一次文件

Question

我將逐行從 SFTP 位置讀取文件：

@Override
public void configure() {
    from(sftpLocationUrl)
            .routeId("route-name")
            .split(body().tokenize("\n"))
            .streaming()
            .bean(service, "build")
            .to(String.format("activemq:%s", queueName));
}

但是這個應用程序將部署在兩個節點上，我認為在這種情況下，我可以獲得一個不穩定且不可預測的應用程序工作，因為文件的相同行可以被讀取兩次。 在這種情況下有沒有辦法避免這種重復？

Answer 1

解決方案是主動被動模式。 “在主動/被動模式下，您有一個主實例輪詢文件，而所有其他實例（從屬）都是被動的。 為了使這種策略起作用，必須使用某種鎖定機制來確保只有持有鎖的節點是主節點，所有其他節點都處於待機狀態。”

它可以用 hazelcast、consul 或 zookeper 實現

public class FileConsumerRoute extends RouteBuilder {

private int delay;
private String name;

public FileConsumerRoute(String name, int delay) {
    this.name = name;
    this.delay = delay;
}

@Override
public void configure() throws Exception {
    // read files from the shared directory
    from("file:target/inbox" +
            "?delete=true")
        // setup route policy to be used
        .routePolicyRef("myPolicy")
        .log(name + " - Received file: ${file:name}")
        .delay(delay)
        .log(name + " - Done file:     ${file:name}")
        .to("file:target/outbox");
}}

服務器欄

public class ServerBar {

private Main main;

public static void main(String[] args) throws Exception {
    ServerBar bar = new ServerBar();
    bar.boot();
}

public void boot() throws Exception {
    // setup the hazelcast route policy
    ConsulRoutePolicy routePolicy = new ConsulRoutePolicy();
    // the service names must be same in the foo and bar server
    routePolicy.setServiceName("myLock");
    routePolicy.setTtl(5);

    main = new Main();
    // bind the hazelcast route policy to the name myPolicy which we refer to from the route
    main.bind("myPolicy", routePolicy);
    // add the route and and let the route be named Bar and use a little delay when processing the files
    main.addRouteBuilder(new FileConsumerRoute("Bar", 100));
    main.run();
}

}

服務器 Foo

public class ServerFoo {

private Main main;

public static void main(String[] args) throws Exception {
    ServerFoo foo = new ServerFoo();
    foo.boot();
}

public void boot() throws Exception {
    // setup the hazelcast route policy
    ConsulRoutePolicy routePolicy = new ConsulRoutePolicy();
    // the service names must be same in the foo and bar server
    routePolicy.setServiceName("myLock");
    routePolicy.setTtl(5);

    main = new Main();
    // bind the hazelcast route policy to the name myPolicy which we refer to from the route
    main.bind("myPolicy", routePolicy);
    // add the route and and let the route be named Bar and use a little delay when processing the files
    main.addRouteBuilder(new FileConsumerRoute("Foo", 100));
    main.run();
}}

資料來源：駱駝行動第二版

Answer 2

Camel 有一些（實驗性的）聚類能力——見這里。

在您的特定情況下，您可以 model 一條在開始目錄輪詢時占據主導地位的路線，從而防止其他節點選擇（相同或其他）文件。