How to set "spark.sql.shuffle.partitions" to auto

Question

My requirement is set the spark.sql.shuffle.partitions=auto

But I'm getting the below error while running the below code

%python
spark.conf.set("spark.sql.shuffle.partitions=auto")

error 
TypeError                                 Traceback (most recent call last)
<command-3238694083679317> in <cell line: 1>()
----> 1 spark.conf.set("spark.sql.shuffle.partitions=auto")

TypeError: set() missing 1 required positional argument: 'value'

Answer 1

Solution is given below.

%python spark.conf.set("spark.sql.shuffle.partitions","auto")

Above code will set the shuffle partitions to "auto"..

And with below code we can see the shuffle partitions value.

%python spark.conf.get("spark.sql.shuffle.partitions")