scala convert into dataframe from Map

Question

How do I convert the input5 data format into dataFrame , using the schema details mentioned in schemanames ..the converion should be dynamic without using Row(r(0),r(1)) the number of columns can increase or decrease in input and schema, hence the code should be dynamic

case class Entry(schemaName: String, updType: String, ts: Long, row: Map[String, String])

val input5 = List(Entry("a","b",0,Map("col1 " -> "0000555", "ref" -> "2017-08-12 12:12:12.266528")))  

val schemanames= "col1,ref"     

Target dataframe should be only from Map of input 5 ..like col 1 and ref there can be many other columns like col2, col3 ... if there are more columns in Map same columns would be mentioned in schema name . Schema name variable should be used to create structure , input5.row(Map) should be data source ...as number of columns in schema name can be in 100's , same applies to data in Input5.row

Answer 1

Here is the code for this

case class Entry(schemaName: String, updType: String, ts: Long, row: Map[String, String])
val input5 = List(Entry("a","b",0,Map("col1 " -> "0000555", "ref" -> "2017-08-12 12:12:12.266528")))
import spark.implicits._
val df = input.toDF 

df will become the dataframe.

Answer 2

You can directly call toDF.

scala> case class Entry(schemaName: String, updType: String, ts: Long, row: Map[String, String])
defined class Entry
scala> val input5 = List(Entry("a","b",0,Map("col1 " -> "0000555", "ref" -> "2017-08-12 12:12:12.266528")))  
input5: List[Entry] = List(Entry(a,b,0,Map(col1  -> 0000555, ref -> 2017-08-12 12:12:12.266528)))

scala> val df = input5.toDF
df: org.apache.spark.sql.DataFrame = [schemaName: string, updType: string ... 2 more fields]

scala> df.show
+----------+-------+---+--------------------+
|schemaName|updType| ts|                 row|
+----------+-------+---+--------------------+
|         a|      b|  0|Map(col1  -> 0000...|
+----------+-------+---+--------------------+