如何使用 pyspark 创建带有 windows 行尾 (CRLF) 的文本文件？

Question

I have a dynamic frame created with the required data. I'm using pyspark in aws glue job. I'm creating a file as below

df_final_data.coalesce.write.format("text").option("mode", "append").save(var_bucket)

But this is creating a file with LF line endings.

How can I create a file with CRLF line endings or is there a way to replace these LF to CRLF after the file is created?

Answer 1

Try using the lineSep parameter:

df_final_data.coalesce.write.format("text").option("lineSep", "\r\n").option("mode", "append").save(var_bucket)