How to convert sql table into a pyspark/python data structure and return back to sql in databricks notebook

Question

I am running a sql notebook on databricks. I would like to analyze a table with half a billion records in it. I can run simple sql queries on the data. However, I need to change the date column type from str to date.

Unfortunately, update/alter statements do not seem to be supported by sparkSQL so it seems I cannot modify the data in the table.

What would be the one-line of code that would allow me to convert the SQL table to a python data structure (in pyspark) in the next cell? Then I could modify the file and return it to SQL.

Answer 1

dataFrame = sqlContext.sql('select * from myTable')

Answer 2

df=sqlContext.sql("select * from table")

要将数据框转换回sql视图，

df.createOrReplaceTempView("myview")