How to save a DataFrame as a csv-file using pyspark?

Question

Why does this approach not work?

from pyspark.sql import SparkSession

spark = SparkSession.builder.appName('session').getOrCreate()
df = spark.range(5).toDF("index")
filepath = r"C:/my_favorite_directory"
df.write.csv(filepath)

---

Update

The above code works fine, the problem was that I had not specified the hadoop binary path to refer to the winutils binary, which is needed by pyspark to write csv-files.

Answer 1

Your filepath should ends with .csv or file extension. You are providing directory thats wrong