计算两组（Java）之间交叉点的最有效方法是什么？

Question

Question Background

I am comparing two (at a time, actually many) text files, and I want to determine how similar they are. To do so, I have created small, overlapping groups of text from each file. I now want to determine the number of those groups from one file which are also from the other file.

I would prefer to use only Java 8 with no external libraries.

Attempts

These are my two fastest methods. The first contains a bunch of logic which allows it to stop if meeting the threshold is not possible with the remaining elements (this saves a bit of time in total, but of course executing the extra logic also takes time). The second is slower. It does not have those optimizations, actually determines the intersection rather than merely counting it, and uses a stream, which is quite new to me.
I have an integer threshold and dblThreshold (the same value cast to a double), which are the minimum percentage of the smaller file which must be shared to be of interest. Also, from my limited testing, it seems that writing all the logic for either set being larger is faster than calling the method again with reversed arguments.

public int numberShared(Set<String> sOne, Set<String> sTwo) {
    int numFound = 0;
    if (sOne.size() > sTwo.size()) {
        int smallSize = sTwo.size();
        int left = smallSize;
        for (String item: sTwo) {
            if (numFound < threshold && ((double)numFound + left < (dblThreshold) * smallSize)) {
                break;
            }
            if (sOne.contains(item)) {
                numFound++;
            }
            left--;
        }
    } else {
        int smallSize = sOne.size();
        int left = smallSize;
        for (String item: sOne) {
            if (numFound < threshold && ((double)numFound + left < (dblThreshold) * smallSize)) {
                break;
            }
            if (sTwo.contains(item)) {
                numFound++;
            }
            left--;
        }
    }
    return numFound;
}

Second method:

public int numberShared(Set<String> sOne, Set<String> sTwo) {
    if (sOne.size() < sTwo.size()) {
        long numFound = sOne.parallelStream()
                            .filter(segment -> sTwo.contains(segment))
                            .collect(Collectors.counting());
        return (int)numFound;
    } else {
        long numFound = sTwo.parallelStream()
                            .filter(segment -> sOne.contains(segment))
                            .collect(Collectors.counting());
        return (int)numFound;
    }
}

Any suggestions for improving upon these methods, or novel ideas and approaches to the problem are much appreciated!

Edit: I just realized that the first part of my threshold check (which seeks to eliminate, in some cases, the need for the second check with doubles) is incorrect. I will revise it as soon as possible.

Answer 1

If I understand you correctly, you have already determined which methods are fastest, but aren't sure how to implement your threshold-check when using Java 8 streams. Here's one way you could do that - though please note that it's hard for me to do much testing without having proper data and knowing what thresholds you're interested in, so take this simplified test case with a grain of salt (and adjust as necessary).

public class Sets {
    private static final int NOT_ENOUGH_MATCHES = -1;
    private static final String[] arrayOne = { "1", "2", "4", "9" };
    private static final String[] arrayTwo = { "2", "3", "5", "7", "9" };
    private static final Set<String> setOne = new HashSet<>();
    private static final Set<String> setTwo = new HashSet<>();

    public static void main(String[] ignoredArguments) {
        setOne.addAll(Arrays.asList(arrayOne));
        setTwo.addAll(Arrays.asList(arrayTwo));
        boolean isFirstSmaller = setOne.size() < setTwo.size();
        System.out.println("Number shared: " + (isFirstSmaller ?
                numberShared(setOne, setTwo) : numberShared(setTwo, setOne)));
    }

    private static long numberShared(Set<String> smallerSet, Set<String> largerSet) {
        SimpleBag bag = new SimpleBag(3, 0.5d, largerSet, smallerSet.size());
        try {
            smallerSet.forEach(eachItem -> bag.add(eachItem));
            return bag.duplicateCount;
        } catch (IllegalStateException exception) {
            return NOT_ENOUGH_MATCHES;
        }
    }

    public static class SimpleBag {
        private Map<String, Boolean> items;
        private int threshold;
        private double fraction;
        protected int duplicateCount = 0;
        private int smallerSize;
        private int numberLeft;

        public SimpleBag(int aThreshold, double aFraction, Set<String> someStrings,
                int otherSetSize) {
            threshold = aThreshold;
            fraction = aFraction;
            items = new HashMap<>();
            someStrings.forEach(eachString -> items.put(eachString, false));
            smallerSize = otherSetSize;
            numberLeft = otherSetSize;
        }

        public void add(String aString) {
            Boolean value = items.get(aString);
            boolean alreadyExists = value != null;
            if (alreadyExists) {
                duplicateCount++;
            }
            items.put(aString, alreadyExists);
            numberLeft--;
            if (cannotMeetThreshold()) {
                throw new IllegalStateException("Can't meet threshold; stopping at "
                        + duplicateCount + " duplicates");
            }
        }

        public boolean cannotMeetThreshold() {
            return duplicateCount < threshold
                    && (duplicateCount + numberLeft < fraction * smallerSize);
        }
    }
}

So I've made a simplified "Bag-like" implementation that starts with the contents of the larger set mapped as keys to false values (since we know there's only one of each). Then we iterate over the smaller set, adding each item to the bag, and, if it's a duplicate, switching the value to true and keeping track of the duplicate count (I initially did a .count() at the end of .stream().allMatch() , but this'll suffice for your special case). After adding each item, we check whether we can't meet the threshold, in which case we throw an exception (arguably not the prettiest way to exit the .forEach() , but in this case it is an illegal state of sorts). Finally, we return the duplicate count, or -1 if we encountered the exception. In my little test, change 0.5d to 0.51d to see the difference.