Databricks save Rdata file to AWS S3 Bucket

Question

I have developed a model in R using databricks. I want to save the output datafile on AWS S3 bucket but when I save the file as below it does not save to the mounted drive.

doc <- save(data, file=paste0(getwd(), "/datafile.RData"))

What is the best way to mount data to S3 using R?

I have tried the below sample code and it works so I know my connection between AWS and Databricks is working.

%python
display(dbutils.fs.ls("/"))

Answer 1

From Databricks File System (DBFS) documentation :

You can use local file APIs to read and write to DBFS paths. Databricks configures each cluster node with a FUSE mount /dbfs that allows processes running on cluster nodes to read and write to the underlying distributed storage layer with local file APIs. When using local file APIs, you must provide the path under /dbfs.

For example:

save(data, file="/dbfs/datafile.RData")