Spark .show()'s table is not aligned in Google Colab

Question

I am testing Apache Spark command on Google Colabs, but the table that is printed out does not aligned. It looks fine on some machines, but not on some machines.

So I wonder if anyone could help clarify why does this happen. Thank you!

Answer 1

Spark has an option to show your dataframe on Jupyter Notebook like Pandas.

spark.conf.set("spark.sql.repl.eagerEval.enabled",True)

Your Dataframe will be shown as a HTML Table.

df = spark.read.option("header",True).csv("sample.csv")
df.limit(10)