How to pass DataFrame as input to Spark UDF?

Question

I have written code in python for eg: environment:Zeppelin

def variabletype(dataframe):
   df = dataframe
   num_col_list=['int','float']
   numerical_columns ##etc
   return cols_list_df

variabletype(df)

I need to apply same function on pyspark dataframe how to do that?

Answer 1

Use the same method you are using currently, for eg

df = spark.createDataFrame(['a', 'a', 'a', 'a', 'a', 'b', 'b', 'b', 'a', 'a', 'b'], 'string').toDF("Data")

def print_cols(df):
    print(df.columns)
    
print_cols(df)

Output:

['Data']