[英]How to remove extra Escape characters from a text column in spark dataframe
My Data in a json looks like -我在 json 中的数据看起来像 -
{"text": "\"I have recently taken out a 12 month mobile phone contract with Virgin but despite two calls to customer help I still am getting a message on my phone indicating \\\"No Service\\\" although intermittently I do get connected.\"", "created_at": "\"2018-08-27 16:58:30\"", "service_id": "51870", "category_id": "249"}
I read this JSON Using -我读了这个 JSON 使用 -
val complaintsSourceRaw = spark.read.json("file:///complaints.jsonl")
When i read the data in dataframe, it looks like当我读取 dataframe 中的数据时,它看起来像
|249 |"2018-08-27 16:58:30"|51870 |"I have recently taken out a 12 month mobile phone contract with Virgin but despite two calls to customer help I still am getting a message on my phone indicating **\"No Service\"** although intermittently I do get connected."
Issue is问题是
**\"No Service\"** need to be like **"No Service"**
How i am trying -我是如何尝试的-
complaintsSourceRaw.withColumn("text_cleaned", functions.regexp_replace(complaintsSourceRaw.col("text"), "\", ""));
However \ character excapes my " and code breaks. Any idea how to acieve this?但是 \ 字符使我的 " 和代码中断。知道如何实现这一点吗?
You need to escape the "\" character, so in your regexp_replace you should look for two backslash ("\\") characters, not one.您需要转义“\”字符,因此在您的 regexp_replace 中您应该寻找两个反斜杠 ("\\") 字符,而不是一个。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.