如何在分布式系統中以相同的方式隨機化兩個字符串 collections？

Question

我有幾個 collections 字符串（約 100k 個元素）。 它們從相同的網絡源初始化：

List<String> source = Arrays.asList("cat Tom", "dog Woof", "bird Carry", "fish Silent", "mice Dr Black");
List<String> list1 = new ArrayList<>(source);
List<String> list2 = new ArrayList<>(source); 
List<String> list3 = new ArrayList<>(source);

在他們的一生中，他們可以獨立修改：

list1.add("raccoon George");
list2.set(2, "bird Frank"); // Different bird
list3.remove(2); // No birds!

這些 collections 存在於不同的 JVM 中，但具有相同（共享）的種子：

String seed = UUID.randomUUID().toString();

我需要一些排序/洗牌方法（或Comparator ）來隨機化這些 collections：

magicSort(list1, seed); // JVM #1
magicSort(list2, seed); // JVM #2
magicSort(list3, seed); // JVM #3

要像這樣接收一致/可重復的 output：

[mice Dr Black, bird Carry, raccoon George, cat Tom, dog Woof, fish Silent]
[mice Dr Black, bird Frank, cat Tom, dog Woof, fish Silent]
[mice Dr Black, cat Tom, dog Woof, fish Silent]

這不起作用：

• 此答案中的Collections.shuffle() - 因為它不為具有相同前綴的項目提供親和力；
• 在Map<String, UUID>中保持隨機順序 - 因為 (a) 集合的動態性質和 (b) 分布式設置。

任何幫助將不勝感激

編輯#1

這不起作用：

• sha1 ， md5和類似的 hash 算法 - 因為它們不為具有相同前綴的項目提供親和力 - bird Carry和bird Frank可以放置在遠離彼此的位置。

編輯#2

附加條件：具有相同前綴的項目應該組合在一起（親和），例如所有的鳥都放在老鼠和浣熊之間。

Answer 1

非常感謝評論中的散列提示！

我最終通過偽隨機散列（類似於 Fisher-Yates shuffle）重新映射字符順序。 rest 代碼是從String.compareTo復制粘貼的：

private void magicSort(List<String> list, String seed) {
    list.sort(new ShuffleComparator(seed));
}

public class ShuffleComparator implements Comparator<String> {
    private static final long MAGIC = 0x5DEECE66DL;
    private final String seed;

    public ShuffleComparator(String seed) {
        this.seed = seed;
    }

    @Override
    public int compare(String s1, String s2) {
        int len1 = s1.length();
        int len2 = s2.length();
        int lim = Math.min(len1, len2);

        int seedLen = seed.length();
        long random = seed.hashCode();

        for (int k = 0; k < lim; k++) {
            random = random * MAGIC + seed.charAt(k % seedLen);
            char c1 = s1.charAt(k);
            char c2 = s2.charAt(k);
            if (c1 != c2) {
                return ((random % (c1 + 0xFFFF)) & 0xFFFF) -
                    ((random % (c2 + 0xFFFF)) & 0xFFFF) > 0 ? 1 : -1;
            }
        }
        return (random > 0 ? 1 : -1) * (len1 - len2);
    }
}

結果：

• 您可以始終保持 collections 隨機排序
• 具有相同前綴的項目組合在一起（親和力）
• JVM 之間共享唯一的資源（字符串種子）
• 對性能的影響可以忽略不計
• 零垃圾回收

例子：

seed: b85a9119-3a98-4491-b503-fc2cfdc344f3
[raccoon George, bird Carry, mice Dr Black, fish Silent, cat Tom, dog Woof]
[bird Frank, mice Dr Black, fish Silent, cat Tom, dog Woof]
[mice Dr Black, fish Silent, cat Tom, dog Woof]

seed: d5378421-d0ea-4b17-80e9-1de6a163bf38
[bird Carry, dog Woof, fish Silent, raccoon George, mice Dr Black, cat Tom]
[bird Frank, dog Woof, fish Silent, mice Dr Black, cat Tom]
[dog Woof, fish Silent, mice Dr Black, cat Tom]

seed: a5f94f60-1207-4f83-9723-bbab5e3b8075
[cat Tom, raccoon George, mice Dr Black, bird Carry, dog Woof, fish Silent]
[cat Tom, mice Dr Black, bird Frank, dog Woof, fish Silent]
[cat Tom, mice Dr Black, dog Woof, fish Silent]

seed: 4b77a61f-c639-42fc-96b2-e1add9f359e9
[bird Carry, raccoon George, cat Tom, mice Dr Black, dog Woof, fish Silent]
[bird Frank, cat Tom, mice Dr Black, dog Woof, fish Silent]
[cat Tom, mice Dr Black, dog Woof, fish Silent]

seed: 2dd78e21-d4ad-4e8a-b139-5d35fe2b0481
[dog Woof, fish Silent, raccoon George, mice Dr Black, bird Carry, cat Tom]
[dog Woof, fish Silent, mice Dr Black, bird Frank, cat Tom]
[dog Woof, fish Silent, mice Dr Black, cat Tom]

編輯#1

對於測試，我還實現了基於哈希的比較器，因此將在此處發布。 以防萬一您不需要親和力（例如我的情況），但需要純洗牌：

private void magicSort(List<String> list, String seed) {
    list.sort(new HashComparator(seed, "SHA-1"));
}

public class HashComparator implements Comparator<String> {
    private static final long MAGIC = 0x5DEECE66DL;
    private final ThreadLocal<MessageDigest> messageDigest;
    private final byte[] seed;
    private final int seedHash;

    public HashComparator(String seed, String algorithm) {
        this.seed = seed.getBytes(ISO_8859_1);
        this.seedHash = seed.hashCode();
        this.messageDigest = ThreadLocal.withInitial(() -> {
            try {
                return MessageDigest.getInstance(algorithm);
            } catch (NoSuchAlgorithmException e) {
                throw new RuntimeException(e.getMessage(), e);
            }
        });
    }

    @Override
    public int compare(String s1, String s2) {
        if (s1.equals(s2)) {
            return 0;
        }
        int diff = getSignature(s1).compareTo(getSignature(s2));
        if (diff != 0) {
            return diff;
        }
        long random = seedHash;
        random = random * MAGIC + s1.hashCode();
        random = random * MAGIC + s2.hashCode();
        return ((random & 1) == 0 ? 1 : -1) * s1.compareTo(s2);

    }

    private ByteBuffer getSignature(String s) {
        MessageDigest digest = messageDigest.get();
        digest.reset();
        digest.update(seed);
        for (int i = 0, size = s.length(); i < size; i++) {
            char c = s.charAt(i);
            digest.update((byte) (c >> 8));
            digest.update((byte) c);
        }
        return ByteBuffer.wrap(digest.digest());
    }
}

結果：

• 您可以始終保持 collections 隨機排序
• 純隨機播放（對具有相同前綴的項目沒有親和力）
• JVM 之間共享唯一的資源（字符串種子）

例子：

8d465fde-9310-4b6c-9709-5cc395deb45e
[raccoon George, mice Dr Black, dog Woof, fish Silent, cat Tom, bird Carry]
[bird Frank, mice Dr Black, dog Woof, fish Silent, cat Tom]
[mice Dr Black, dog Woof, fish Silent, cat Tom]

6daa90dd-f7e7-4615-a45c-fefb0de10c20
[bird Carry, mice Dr Black, fish Silent, cat Tom, dog Woof, raccoon George]
[mice Dr Black, bird Frank, fish Silent, cat Tom, dog Woof]
[mice Dr Black, fish Silent, cat Tom, dog Woof]

9f30b259-c8d1-41f5-8221-40b17cb04260
[cat Tom, mice Dr Black, raccoon George, dog Woof, fish Silent, bird Carry]
[bird Frank, cat Tom, mice Dr Black, dog Woof, fish Silent]
[cat Tom, mice Dr Black, dog Woof, fish Silent]

94b6710f-3232-453d-8009-6d81658b2cca
[dog Woof, fish Silent, bird Carry, cat Tom, mice Dr Black, raccoon George]
[dog Woof, fish Silent, cat Tom, mice Dr Black, bird Frank]
[dog Woof, fish Silent, cat Tom, mice Dr Black]