Spark's call function

Question

Sorry, basic question about Spark. Can I use Java object different than primitive type inside a Spark's call function? For instance, imagine that I have something like that:

JavaRDD<String> input = sc.textFile(dataFile);
    JavaRDD<String> output;
    output = input.map(new Function<String, String>() { public String call(String s) throws MalformedURLException {
            SystemConfiguration config = new SystemConfiguration();

....

If I remove the instance of my own class SystemConfiguratin it works fine, but with it inside it doesn't work (Spark finished with failure). Please could you shed light about this? Many thanks

Answer 1

只要您在转换中创建的对象是可序列化的或声明为@transient lazy（对于不可序列化的对象），就可以了。