Convert negative decimal number to positive in java spark dataset

Question

I have the below dataframe.

column_1
2.1
-3.4
-1.1
2.5

I want to convert this column to positive numbered column when the value is less than 0 or in other words if its negative value. Input is in decimal format and output should also be in decimal format.

I tried converting to string and using abs(column_1) when lt(0) and then cast it as decimal. But all values are getting converted to null.

Any idea how to achieve this in Java spark.

Required output:

column_1
2.1
3.4
1.1
2.5

Answer 1

you can simply use abs , which is available in Pyspark via the functions package [Docs] but also can be used via the SQL API [Docs] .

Pyspark

from pyspark.sql import functions as F
df = df.withColumn('column_1', F.abs('column_1')

Spark SQL

SELECT abs(column_1)
FROM df;