列是 uuid 类型，但表达式是 Spark Scala 中不同的字符类型

Question

Good day. I am deploying a streaming job to insert data from Spark (Scala) to Postgres.

df.select("col1","col2").write.mode(SaveMode.Append).jdbc(url, "tableName", connectionProperties)

Here col2 is having uuid values in the dataframe df, but it is a string datatype. When it tries to insert in to the table that has col2 column defined as type uuid its failing with the Column is of type uuid but expression is of type character varying . Can someone help me with a workaround for this. ?

Thank you.

Answer 1

When you have column column in dataframe that has string data type but had values of type UUID then you can set the specific property in your connection properties and that will allow you to save your dataframe. If you don't set that property it will not allow you to save the data in the sql/postgres table, if the column has datatype of UUID.

You can set this property in the Connection Properties

connectionProps.setProperty("stringtype", "unspecified")

Setting the above property will allow you to save the string type column having UUID values in the dataframe to a column of type UUID in the postgres database.