使用java流并行收集（供应商，累加器，组合器）不给出预期的结果

Question

I'm trying to find number of words in given string. Below is sequential algorithm for it which works fine.

public int getWordcount() {

        boolean lastSpace = true;
        int result = 0;

        for(char c : str.toCharArray()){
            if(Character.isWhitespace(c)){
                lastSpace = true;
            }else{
                if(lastSpace){
                    lastSpace = false;
                    ++result;
                }
            }
        }

        return result;

    }

But, when i tried to 'parallelize' this with Stream.collect(supplier, accumulator, combiner) method, i am getting wordCount = 0. I am using an immutable class (WordCountState) just to maintain the state of word count.

Code :

public class WordCounter {
    private final String str = "Java8 parallelism  helps    if you know how to use it properly.";

public int getWordCountInParallel() {
        Stream<Character> charStream = IntStream.range(0, str.length())
                                                .mapToObj(i -> str.charAt(i));

        WordCountState finalState = charStream.parallel()                                             
                                              .collect(WordCountState::new,
                                                        WordCountState::accumulate,
                                                        WordCountState::combine);

        return finalState.getCounter();
    }
}

public class WordCountState {
    private final boolean lastSpace;
    private final int counter;
    private static int numberOfInstances = 0;

public WordCountState(){
        this.lastSpace = true;
        this.counter = 0;
        //numberOfInstances++;
    }

    public WordCountState(boolean lastSpace, int counter){
        this.lastSpace = lastSpace;
        this.counter = counter;
        //numberOfInstances++;
    }

//accumulator
    public WordCountState accumulate(Character c) {


        if(Character.isWhitespace(c)){
            return lastSpace ? this : new WordCountState(true, counter);
        }else{
            return lastSpace ? new WordCountState(false, counter + 1) : this;
        }   
    }

    //combiner
    public WordCountState combine(WordCountState wordCountState) {  
        //System.out.println("Returning new obj with count : " + (counter + wordCountState.getCounter()));
        return new WordCountState(this.isLastSpace(), 
                                    (counter + wordCountState.getCounter()));
    }

I've observed two issues with above code : 1. Number of objects (WordCountState) created are greater than number of characters in the string. 2. Result is always 0. 3. As per accumulator/consumer documentation, shouldn't the accumulator return void? Even though my accumulator method is returning an object, compiler doesn't complain.

Any clue where i might have gone off track?

UPDATE : Used solution as below -

public int getWordCountInParallel() {
        Stream<Character> charStream = IntStream.range(0, str.length())
                                                .mapToObj(i -> str.charAt(i));


        WordCountState finalState = charStream.parallel()
                                              .reduce(new WordCountState(),
                                                        WordCountState::accumulate,
                                                        WordCountState::combine);

        return finalState.getCounter();
    }

Answer 1

You can always invoke a method and ignore its return value, so it's logical to allow the same when using method references. Therefore, it's no problem creating a method reference to a non- void method when a consumer is required, as long as the parameters match.

What you have created with your immutable WordCountState class, is a reduction operation, ie it would support a use case like

Stream<Character> charStream = IntStream.range(0, str.length())
                                        .mapToObj(i -> str.charAt(i));

WordCountState finalState = charStream.parallel()
        .map(ch -> new WordCountState().accumulate(ch))
        .reduce(new WordCountState(), WordCountState::combine);

whereas the collect method supports the mutable reduction , where a container instance (may be identical to the result) gets modified.

There is still a logical error in your solution as each WordCountState instance starts with assuming to have a preceding space character, without knowing the actual situation and no attempt to fix this in the combiner.

A way to fix and simplify this, still using reduction, would be:

public int getWordCountInParallel() {
    return str.codePoints().parallel()
        .mapToObj(WordCountState::new)
        .reduce(WordCountState::new)
        .map(WordCountState::getResult).orElse(0);
}


public class WordCountState {
    private final boolean firstSpace, lastSpace;
    private final int counter;

    public WordCountState(int character){
        firstSpace = lastSpace = Character.isWhitespace(character);
        this.counter = 0;
    }

    public WordCountState(WordCountState a, WordCountState b) {
        this.firstSpace = a.firstSpace;
        this.lastSpace = b.lastSpace;
        this.counter = a.counter + b.counter + (a.lastSpace && !b.firstSpace? 1: 0);
    }
    public int getResult() {
        return counter+(firstSpace? 0: 1);
    }
}

If you are worrying about the number of WordCountState instances, note how many Character instances this solution does not create, compared to your initial approach.

However, this task is indeed suitable for mutable reduction, if you rewrite your WordCountState to a mutable result container:

public int getWordCountInParallel() {
    return str.codePoints().parallel()
        .collect(WordCountState::new, WordCountState::accumulate, WordCountState::combine)
        .getResult();
}


public class WordCountState {
    private boolean firstSpace, lastSpace=true, initial=true;
    private int counter;

    public void accumulate(int character) {
        boolean white=Character.isWhitespace(character);
        if(lastSpace && !white) counter++;
        lastSpace=white;
        if(initial) {
            firstSpace=white;
            initial=false;
        }
    }
    public void combine(WordCountState b) {
        if(initial) {
            this.initial=b.initial;
            this.counter=b.counter;
            this.firstSpace=b.firstSpace;
            this.lastSpace=b.lastSpace;
        }
        else if(!b.initial) {
            this.counter += b.counter;
            if(!lastSpace && !b.firstSpace) counter--;
            this.lastSpace = b.lastSpace;
        }
    }
    public int getResult() {
        return counter;
    }
}

Note how using int to represent unicode characters consistently, allows to use the codePoint() stream of a CharSequence , which is not only simpler, but also handles characters outside the Basic Multilingual Plane and is potentially more efficient, as it doesn't need boxing to Character instances.

Answer 2

When you implemented stream().collect(supplier, accumulator, combiner) they do return void (combiner and accumulator). The problem is that this:

  collect(WordCountState::new,
          WordCountState::accumulate,
          WordCountState::combine)

In your case actually means (just the accumulator, but same goes for the combiner):

     (wordCounter, character) -> {
              WordCountState state = wc.accumulate(c);
              return;
     }

And this is not trivial to get indeed. Let's say we have two methods:

public void accumulate(Character c) {
    if (!Character.isWhitespace(c)) {
        counter++;
    }
}

public WordCountState accumulate2(Character c) {
    if (Character.isWhitespace(c)) {
        return lastSpace ? this : new WordCountState(true, counter);
    } else {
        return lastSpace ? new WordCountState(false, counter + 1) : this;
    }
}

For the them the below code will work just fine , BUT only for a method reference , not for lambda expressions.

BiConsumer<WordCountState, Character> cons = WordCountState::accumulate;

BiConsumer<WordCountState, Character> cons2 = WordCountState::accumulate2;

You can imagine it slightly different, via an class that implementes BiConsumer for example:

 BiConsumer<WordCountState, Character> clazz = new BiConsumer<WordCountState, Character>() {
        @Override
        public void accept(WordCountState state, Character character) {
            WordCountState newState = state.accumulate2(character);
            return;
        }
    };

As such your combine and accumulate methods needs to change to:

public void combine(WordCountState wordCountState) {
    counter = counter + wordCountState.getCounter();
}


public void accumulate(Character c) {
    if (!Character.isWhitespace(c)) {
        counter++;
    }
}

Answer 3

First of all, would it not be easier to just use something like input.split("\\\\s+").length to get the word count?

In case this is an exercise in streams and collectors, let's discuss your implementation. The biggest mistake was pointed out by you already: Your accumulator and combiner should not return new instances. The signature of collect tells you that it expects BiConsumer , which do not return anything. Because you create new object in the accumulator, you never increase the count of the WordCountState objects your collector actually uses. And by creating a new object in the combiner you would discard any progress you would have made. This is also why you create more objects than characters in your input: one per character, and then some for the return values.

See this adapted implementation:

public static class WordCountState
{
    private boolean lastSpace = true;
    private int     counter   = 0;

    public void accumulate(Character character)
    {
        if (!Character.isWhitespace(character))
        {
            if (lastSpace)
            {
                counter++;
            }
            lastSpace = false;
        }
        else
        {
            lastSpace = true;
        }
    }

    public void combine(WordCountState wordCountState)
    {
        counter += wordCountState.counter;
    }
}

Here, we do not create new objects in every step, but change the state of the ones we have. I think you tried to create new objects because your Elvis operators forced you to return something and/or you couldn't change the instance fields as they are final. They do not need to be final, though, and you can easily change them.

Running this adapted implementation sequentially now works fine, as we nicely look at the chars one by one and end up with 11 words.

In parallel, though, it fails. It seems it creates a new WordCountState for every char, but does not count all of them, and ends up at 29 (at least for me). This shows a basic flaw with your algorithm: Splitting on every character doesn't work in parallel. Imagine the input abc abc , which should result in 2. If you do it in parallel and do not specify how to split the input, you might end up with these chunks: ab, ca, bc , which would add up to 4.

The problem is that by parallelizing between characters (ie in the middle of words), you make your separate WordCountState s dependent on each other (because they would need to know which one come before them and whether it ended with a whitespace char). This defeats the parallelism and results in errors.

Aside from all that, it might be easier to implement the Collector interface instead of providing the three methods:

public static class WordCountCollector
    implements Collector<Character, SimpleEntry<AtomicInteger, Boolean>, Integer>
{
    @Override
    public Supplier<SimpleEntry<AtomicInteger, Boolean>> supplier()
    {
        return () -> new SimpleEntry<>(new AtomicInteger(0), true);
    }

    @Override
    public BiConsumer<SimpleEntry<AtomicInteger, Boolean>, Character> accumulator()
    {
        return (count, character) -> {
            if (!Character.isWhitespace(character))
            {
                if (count.getValue())
                {
                    String before = count.getKey().get() + " -> ";
                    count.getKey().incrementAndGet();
                    System.out.println(before + count.getKey().get());
                }
                count.setValue(false);
            }
            else
            {
                count.setValue(true);
            }
        };
    }

    @Override
    public BinaryOperator<SimpleEntry<AtomicInteger, Boolean>> combiner()
    {
        return (c1, c2) -> new SimpleEntry<>(new AtomicInteger(c1.getKey().get() + c2.getKey().get()), false);
    }

    @Override
    public Function<SimpleEntry<AtomicInteger, Boolean>, Integer> finisher()
    {
        return count -> count.getKey().get();
    }

    @Override
    public Set<java.util.stream.Collector.Characteristics> characteristics()
    {
        return new HashSet<>(Arrays.asList(Characteristics.CONCURRENT, Characteristics.UNORDERED));
    }
}

We use a pair ( SimpleEntry ) to keep the count and the knowledge about the last space. This way, we do not need to implement the state in the collector itself or write a param object for it. You can use this collector like this:

return charStream.parallel().collect(new WordCountCollector());

This collector parallelizes nicer than the initial implementation, but still varies in results (mostly between 14 and 16) because of the mentioned weaknesses in your approach.