如何使用 scala 删除文件中的特殊字符？

Question

我有一个文本文件，我想从每一行的开头和结尾删除特殊字符“[”，“]”。

[65,898,"screwball comedy"]
[121,778,"dark comedy"]
[121,1288,"comedy"]
[121,2706,"teen comedy"]
[121,35836,"comedy"]
[121,35836,"sex comedy"]
[121,36529,"dark comedy"]

如何从所有文件中删除“[”和“]”？

Answer 1

您可以使用trim function：

val df = spark.read.text(path)
val df1 = df.withColumn("value", expr("trim(both '][' from value)"))

df1.show(false)
//+-------------------------+
//|value                    |
//+-------------------------+
//|65,898,"screwball comedy"|
//|121,778,"dark comedy"    |
//|121,1288,"comedy"        |
//|121,2706,"teen comedy"   |
//|121,35836,"comedy"       |
//|121,35836,"sex comedy"   |
//|121,36529,"dark comedy"  |
//+-------------------------+

或使用regexp_replace ：

val df1 = df.withColumn("value", regexp_replace(col("value"), "[\\[\\]]", ""))