cast string column to decimal in scala dataframe

Question

I have a dataframe (scala) I am using both pyspark and scala in a notebook

#pyspark
spark.read.csv(output_path + '/dealer', header = True).createOrReplaceTempView('dealer_dl')

%scala
import org.apache.spark.sql.functions._
val df = spark.sql("select * from dealer_dl")

How to convert a string column (amount) into decimal in scala dataframe.

I tried as below.

 %scala
 df = df.withColumn("amount", $"amount".cast(DecimalType(9,2)))

But I am getting an error as below:

error: reassignment to val

I am used to pyspark and quite new to scala. I need to do by scala to proceed further. Please let me know. Thanks.

Answer 1

in scala you can't reasign references defined as val but val is immutable reference. if you want to use reasigning some ref you can use var but better solution is not reasign something to the same reference name and use another val . For example:

val dfWithDecimalAmount = df.withColumn("amount", $"amount".cast(DecimalType(9,2)))