Null Pointer Exception in Hadoop Reduce method?

Question

I'm building an inverted index and currently getting a null pointer exception in reduce when using context.write. Can anyone spot why? I presume something to do with serialising as I've never done that before? The error also happens when I print out h.

Answer 1

Two things that I can spot directly regarding the serialization without a stacktrace:

1. HMapValue needs a default constructor, it can't be created by Hadoop without one
2. In the default constructor you need to initialize the ArrayListWritable correctly (not null and it needs the setClass method to deserialize correctly.

Answer 2

Turns out it was because I had iterated over the data set twice (the line int df = Iterables.size(values); tricked me). The iterator hadn't reset hence the main block of reduce didnt run and finally I hit a null pointer because I tried accessing my data that hadn't even initialised.