How to call spark UDF from scala using reflection?

Question

I am building a spark application which has dependency on a java library. Java Interface exposed is as

String doSomething(String, Map<String,String>)

I have created a UDF as

def myfunc(properties: Map[String, String]) = udf((data: String) => {
    ...
    doSomething(data,properties)
})

This function can be called as myfunc(properties)(data) from spark shell where properties is a Map and data is of type Column.

The issue is I need to call this via reflection from a scala file. I need to do something like this:

val c = Class.forName("package.class")
val m = c.getMethod("myfunc",classOf[Map[String,String]])
m.invoke(c.newInstance, someMap)

m.invoke returns the function itself. How and where to pass the Column parameter? Or is there any other way to pass these properties map to spark UDF so that it can directly be called via reflection?

Answer 1

Try

m.invoke(c.newInstance, someMap).asInstanceOf[UserDefinedFunction].apply(data)

for data of type Column .