如何将dataFrame csv/excel文件导出到特定目录

Question

Databricks dataFrame

I have a dataFrame that I would like to export the result to a specific path/directory.

By doing display(df) , you got a button that you can press that will give you a download off the result that you can open in excel.

How can I do this by using python? I am trying to make a job in databricks that run in a specific schedule, and I want it to run my entire notebook, and in the end make it export the final result and save it in a specific directory so I can go back in time and see some old result.

Answer 1

You can export the data to a CSV format(Which can be opened in Excel). In order to do so you can convert the DataFrame to a Pandas DataFrame and then convert it to CSV format.

df.toPandas().to_csv("<FILE_DESTINATION>/<FILE_NAME>")

Answer 2

From what I understand, you want to know how to export a pandas dataframe(csv/ excel) to a directory:

You can use:

import pandas
specific_dir = '/path_of_dir'

df.to_csv(specific_dir+'/mycsv.csv')
df.to_excel(specific_dir+'/myexcel.xlsx')

You can just pass the directory you want to export the dataframe into in the commands above.