How to iterate nested lists with lambda streams?

Question

I'm trying to refactor the following code to lambda expressions with `stream, especially the nested foreach loops:

public static Result match (Response rsp) {
    Exception lastex = null;

    for (FirstNode firstNode : rsp.getFirstNodes()) {
        for (SndNode sndNode : firstNode.getSndNodes()) {
            try {
                if (sndNode.isValid())
                return parse(sndNode); //return the first match, retry if fails with ParseException
            } catch (ParseException e) {
                lastex = e;
            }
        }
    }

    //throw the exception if all elements failed
    if (lastex != null) {
        throw lastex;
    }

    return null;
}

I'm starting with:

rsp.getFirstNodes().forEach().?? // how to iterate the nested 2ndNodes?

Answer 1

Look at flatMap:

flatMap(Function<? super T,? extends Stream<? extends R>> mapper)
Returns a stream consisting of the results of replacing each element of this stream with the contents of a mapped stream produced by applying the provided mapping function to each element.

Code sample assuming isValid() doesn't throw

Optional<SndNode> sndNode = rsp.getFirstNodes()
  .stream()
  .flatMap(firstNode -> firstNode.getSndNodes().stream())  //This is the key line for merging the nested streams
  .filter(sndNode -> sndNode.isValid())
  .findFirst();

if (sndNode.isPresent()) {
    try {
        parse(sndNode.get());
    } catch (ParseException e) {
        lastex = e;
    }
}

Answer 2

I am afraid that using streams and lambdas, your performance may suffer. Your current solution returns the first valid and parse-able node, however it is not possible to interrupt an operation on stream such as for-each ( source ).

Also, because you can have two different outputs (returned result or thrown exception), it won't be possible to do this with single line expression.

Here is what I came up with. It may give you some ideas:

public static Result match(Response rsp) throws Exception {
    Map<Boolean, List<Object>> collect = rsp.getFirstNodes().stream()
            .flatMap(firstNode -> firstNode.getSndNodes().stream()) // create stream of SndNodes
            .filter(SndNode::isValid) // filter so we only have valid nodes
            .map(node -> {
                // try to parse each node and return either the result or the exception
                try {
                    return parse(node);
                } catch (ParseException e) {
                    return e;
                }
            }) // at this point we have stream of objects which may be either Result or ParseException
            .collect(Collectors.partitioningBy(o -> o instanceof Result)); // split the stream into two lists - one containing Results, the other containing ParseExceptions

    if (!collect.get(true).isEmpty()) {
        return (Result) collect.get(true).get(0);
    }
    if (!collect.get(false).isEmpty()) {
        throw (Exception) collect.get(false).get(0); // throws first exception instead of last!
    }
    return null;
}

As mentioned at the beginning, there is possible performance issue as this will try to parse every valid node .

---

EDIT:

To avoid parsing all nodes, you could use reduce , but it is a bit more complex and ugly (and extra class is needed). This also shows all ParseException s instead of just last one.

private static class IntermediateResult {

    private final SndNode node;
    private final Result result;
    private final List<ParseException> exceptions;

    private IntermediateResult(SndNode node, Result result, List<ParseException> exceptions) {
        this.node = node;
        this.result = result;
        this.exceptions = exceptions;
    }

    private Result getResult() throws ParseException {
        if (result != null) {
            return result;
        }
        if (exceptions.isEmpty()) {
            return null;
        }
        // this will show all ParseExceptions instead of just last one
        ParseException exception = new ParseException(String.format("None of %s valid nodes could be parsed", exceptions.size()));
        exceptions.stream().forEach(exception::addSuppressed);
        throw exception;
    }

}

public static Result match(Response rsp) throws Exception {
    return Stream.concat(
                    Arrays.stream(new SndNode[] {null}), // adding null at the beginning of the stream to get an empty "aggregatedResult" at the beginning of the stream
                    rsp.getFirstNodes().stream()
                            .flatMap(firstNode -> firstNode.getSndNodes().stream())
                            .filter(SndNode::isValid)
            )
            .map(node -> new IntermediateResult(node, null, Collections.<ParseException>emptyList()))
            .reduce((aggregatedResult, next) -> {
                if (aggregatedResult.result != null) {
                    return aggregatedResult;
                }

                try {
                    return new IntermediateResult(null, parse(next.node), null);
                } catch (ParseException e) {
                    List<ParseException> exceptions = new ArrayList<>(aggregatedResult.exceptions);
                    exceptions.add(e);
                    return new IntermediateResult(null, null, Collections.unmodifiableList(exceptions));
                }
            })
            .get() // aggregatedResult after going through the whole stream, there will always be at least one because we added one at the beginning
            .getResult(); // return Result, null (if no valid nodes) or throw ParseException
}

---

EDIT2:

In general, it is also possible to use lazy evaluation when using terminal operators such as findFirst() . So with a minor change of requirements (ie returning null instead of throwing exception), it should be possible to do something like below. However, flatMap with findFirst doesn't use lazy evaluation ( source ), so this code tries to parse all nodes.

private static class ParsedNode {
    private final Result result;

    private ParsedNode(Result result) {
        this.result = result;
    }
}

public static Result match(Response rsp) throws Exception {
    return rsp.getFirstNodes().stream()
            .flatMap(firstNode -> firstNode.getSndNodes().stream())
            .filter(SndNode::isValid)
            .map(node -> {
                try {
                    // will parse all nodes because of flatMap
                    return new ParsedNode(parse(node));
                } catch (ParseException e ) {
                    return new ParsedNode(null);
                }
            })
            .filter(parsedNode -> parsedNode.result != null)
            .findFirst().orElse(new ParsedNode(null)).result;
}

Answer 3

Try to use map which transform the original source.

   rsp.getFirstNodes().stream().map(FirstNode::getSndNodes)
               .filter(sndNode-> sndNode.isValid())
               .forEach(sndNode->{
   // No do the sndNode parsing operation Here.
   })

Answer 4

You can iterate nested loops like below

allAssessmentsForJob.getBody().stream().forEach(assessment -> {
        jobAssessments.stream().forEach(jobAssessment -> {
            if (assessment.getId() == jobAssessment.getAssessmentId()) {
                jobAssessment.setAssessment(assessment);
            }
        });
    });

Answer 5

A little bit late but here is a readable approach:

   Result = rsp.getFirstNodes()
        .stream()
        .flatMap(firstNode -> firstNode.getSndNodes.stream())
        .filter(secondNode::isValid))
        .findFirst()
        .map(node -> this.parseNode(node)).orElse(null);

Explanation : you get all the firstNodes and stream() them up. Out a each firstNode you bring n SndNodes . You check each SndNodes to see find the first one that is valid . If there is no valid SndNode then we'll get a null. If there is one, it'll get parsed into a Result

the parseMethod() doesn't change from the original:

public Result parseNode(SndNode node){
        try {
        ...
        ... // attempt to parsed node 
    } catch (ParseException e) {
        throw new ParseException;
    }   
} 

Answer 6

You could use that fact that StreamSupport provides a stream method that takes a Spliterator and Iterable has a spliterator method.

You then just need a mechanism to flatten your structure into an Iterable - something like this.

class IterableIterable<T> implements Iterable<T> {

    private final Iterable<? extends Iterable<T>> i;

    public IterableIterable(Iterable<? extends Iterable<T>> i) {
        this.i = i;
    }

    @Override
    public Iterator<T> iterator() {
        return new IIT();
    }

    private class IIT implements Iterator<T> {

        // Pull an iterator.
        final Iterator<? extends Iterable<T>> iit = i.iterator();
        // The current Iterator<T>
        Iterator<T> it = null;
        // The current T.
        T next = null;

        @Override
        public boolean hasNext() {
            boolean finished = false;
            while (next == null && !finished) {
                if (it == null || !it.hasNext()) {
                    if (iit.hasNext()) {
                        it = iit.next().iterator();
                    } else {
                        finished = true;
                    }
                }
                if (it != null && it.hasNext()) {
                    next = it.next();
                }
            }
            return next != null;
        }

        @Override
        public T next() {
            T n = next;
            next = null;
            return n;
        }
    }

}

public void test() {
    List<List<String>> list = new ArrayList<>();
    List<String> first = new ArrayList<>();
    first.add("First One");
    first.add("First Two");
    List<String> second = new ArrayList<>();
    second.add("Second One");
    second.add("Second Two");
    list.add(first);
    list.add(second);
    // Check it works.
    IterableIterable<String> l = new IterableIterable<>(list);
    for (String s : l) {
        System.out.println(s);
    }
    // Stream it like this.
    Stream<String> stream = StreamSupport.stream(l.spliterator(), false);
}

You can now stream directly from your Iterable .

Initial research suggests that this should be done with flatMap but whatever.