'DataFrame' object has no attribute 'copy'

Question

I have a dataframe and I want to make a deep copy of it so I can modify the copy and use it in further processing. I am working in Azure Databricks.

My dataframe is called "a" and I tried the following command:

b = a.copy(deep=True)

When I run it, I encounter the following error :

'DataFrame' object has no attribute 'copy'

I also tried to use 'iloc' or 'loc' function to create a new dataframe only with the columns that I need, but same error ('DataFrame' object has no attribute 'lit').

Any ideas why is this happening?..

Answer 1

Assuming you're working in Python, check whether you're using a Spark DataFrame or a pandas DataFrame . If you're using a pandas one then I couldn't tell you what's going on without more information; if you're using the spark one then you should use

newDataFrame = oldDataFrame.select('*')