Scala-Spark - Try/Catch is changing type of case class

Question

I am trying to use the sellmerfud.optparse to parse the parameters used to launch the Spark2-Submit jar file. The code I am pasting below is adapted to paste into the spark2-shell where I force the args values as they would be passed via the command line.

The problem is that when I use the try/catch the variable I am setting does not get assigned to the class but when I do not use the try/catch, the args array parses fine and the values are available I am pasting what I captured in the spark2-shell session to provide information to see what is happening.

  import org.sellmerfud.optparse._
  case class Config(keyVal: String            = "tst",
                  startDate: String          = "",
                  endDate: String            = "")
  var args = Array("-k","testkey","-s","2018-01-01","-e","2018-03-31")
  var clsCfg = try {
    new OptionParser[Config] {
      banner = "testargs [options] file...\n"
      separator("")
      separator("Options: ")
      reqd[String]("-k <string>", "--keyval <string>", "Enter Key Value")
          { (v, cfg) => cfg.copy(keyVal = v) }
      optl[String]("-s", "--startdate <date>", "Enter date in mm-dd-yyyy format.")
          { (v, cfg) => cfg.copy(startDate = v.toString()) }
      optl[String]("-e", "--enddate <date>", "Enter date in mm-dd-yyyy format.")
          { (v, cfg) => cfg.copy(endDate = v.toString) }
    }.parse(args, Config())
  } catch { case e: OptionParserException => println(e.getMessage); java.lang.System.exit(1) }
  println("clsCfg: " + clsCfg)
  println("\nArguments passed as array, one array element per row:")
  println(args.deep.mkString("\n"))
  println("clscfg.keyVal: " + clsCfg.keyVal)

Lines output from above code:

import org.sellmerfud.optparse._
defined class Config
args: Array[String] = Array(-k, testkey, -s, 2018-01-01, -e, 2018-03-31)
clsCfg: Any = Config(testkey,Some(2018-01-01),Some(2018-03-31))
clsCfg: Config(testkey,Some(2018-01-01),Some(2018-03-31))
Arguments passed as array, one array element per row:
-k
testkey
-s
2018-01-01
-e
2018-03-31
<console>:42: error: value keyVal is not a member of Any
           println("clscfg.keyVal" + clsCfg.keyVal)

But when I remove the try/catch lines as follows:

  import org.sellmerfud.optparse._
  case class Config(keyVal: String            = "tst",
                  startDate: String          = "",
                  endDate: String            = "")
  var args = Array("-k","testkey","-s","2018-01-01","-e","2018-03-31")
  var clsCfg =
    new OptionParser[Config] {
      banner = "testargs [options] file...\n"
      separator("")
      separator("Options: ")
      reqd[String]("-k <string>", "--keyval <string>", "Enter Key Value")
          { (v, cfg) => cfg.copy(keyVal = v) }
      optl[String]("-s", "--startdate <date>", "Enter date in mm-dd-yyyy format.")
          { (v, cfg) => cfg.copy(startDate = v.toString()) }
      optl[String]("-e", "--enddate <date>", "Enter date in mm-dd-yyyy format.")
          { (v, cfg) => cfg.copy(endDate = v.toString) }
    }.parse(args, Config())

  println("clsCfg: " + clsCfg)
  println("\nArguments passed as array, one array element per row:")
  println(args.deep.mkString("\n"))
  println("clscfg.keyVal: " + clsCfg.keyVal)

I get these results which does parse the string BUT More importantly it creates the variable clsCfg where I can access the class members easily.

import org.sellmerfud.optparse._
defined class Config
args: Array[String] = Array(-k, testkey, -s, 2018-01-01, -e, 2018-03-31)
clsCfg: Config = Config(testkey,Some(2018-01-01),Some(2018-03-31))
clsCfg: Config(testkey,Some(2018-01-01),Some(2018-03-31))
Arguments passed as array, one array element per row:
-k
testkey
-s
2018-01-01
-e
2018-03-31
clscfg.keyVal: testkey   

I think that I need the try/catch to properly handle malformed variables handed in, but I don't know how to get that working. Any help is appreciated.

Answer 1

What's happening can be shown in an example. Basically, when you use try .. catch , scala will look at each return type of all the code blocks ( try and all the catch , since there may be multiple) and return the one type that is common to all of them. In your case, it's Any . Example:

val a  =  try { 1 } // a will be of type Int
val b =   try { 1 } catch { _ => 2 } // b will be of type Int
val c =   try { 1 } catch { _ => "blah" } // c will be of type Any

If you want a more scala-ish way to handle it, you could do as somebody is suggesting in the comments and use the Try() monad, but you could build a function that returns Option[Config]

def buildConfig(args:Array[String]):Option[Config] = try { 
 Some(new OptionParser[Config] {
  ....
  }.parse(args, Config())
} catch { _ => None }   

Note that both try and catch return a Option , since Some() and None are both an Option . Then use pattern matching to make sure the processing was successful

buildConfig(args) match {
     case Some(conf) => // normal processing
     case None => //
}

Answer 2

As mentioned in another answer, the return type of your try statement is the common type among the results of the try and all catch blocks. Your try results in a Config , but your catch results in a Unit (as it doesn't return any value), and their common type is Any . You can try adding an explicit return type to your function, and it will fail compilation (implicit typing, wile useful in general, tends to hide errors like this).

You can either return an Option[Config] type (with the catch block returning None , and the normal processing returning Some(config) ), or, if you'd like to preserve the exception information, you can use Either type:

val clsCfg: Either[Config, Throwable] = try {
  ...
  Left(config)
} catch  { case e: OptionParserException => Right(e) }

which you can later match on:

clsCfg match {
  case Left(config) => // use your config here
  case Right(exception) => // handle exception
}