Spark in Kryo : “java.lang.StackOverflowErrorat com.esotericsoftware.kryo.util.IdentityObjectIntMap.put”

Question

everyone. I met the bug shown below:

I noticed that it just refer a map method in scala, I wonder how to solve this problem. Thank you very much~

Answer 1

OK, I will solve my problem by myself. Actually, the reason why the stackoverflow happens is that I use the Mutable.List in my code. However, List has the maximum value of length. While the number of element you want to put into List exceeds the maximum value of List designed length, this bug occurs. So the solution is to check your code and to see whether you have used some data structure which might has its limits in length or not.