databricks python dbutils can't move file from one directory to another

Question

I have a file that I can see in my current working directory:

%sh
pwd
ls

The output of above is:

/databricks/driver
conf
sample.csv
logs

I want to move the sample.csv file from here to Workspace/Shared directory for which I am using dbutils.fs.mv :

dbutils.fs.mv("dbfs:/databricks/driver/sample.csv","dbfs:/Workspace/Shared/")

but this gives error as java.is.FileNotFoundException:dbfs:/databricks/driver/sample.csv

How do I resolve this error?

Answer 1

when you're executing command on via %sh , it's executed on the driver node, so file is local to it. But you're trying to copy file as it's on the DBFS already, and then it isn't found. You need to change scheme from dbfs to file to point to the file on the driver node, like this:

dbutils.fs.mv("file:///databricks/driver/sample.csv","dbfs:/Workspace/Shared/")