Hope somebody can help me with this as I'm completely out of ideas as to why it's happening. I am currently conducting some analysis on Premier Leag ...
Hope somebody can help me with this as I'm completely out of ideas as to why it's happening. I am currently conducting some analysis on Premier Leag ...
I have a DataFrame that has multiple columns of which some of them are structs. Something like this I want to apply a UserDefinedFunction on the co ...
I would like to build one UDF from two already working functions. I'm trying to calculate a md5 hash as a new column to an existing Spark Dataframe. ...
I'm trying to write an UDF that I would like to use on Hive tables in an sqlContext. Is it in any way possible to include objects from other libraries ...
When the user enters too many arguments for the COUNTBLANK function,the function displays this error message, and returns to edit mode: You've ent ...
I have 200 Mil rows with 1K groups looking like this I want to run the same function (say linear regression of X on [X, Z, Q, W]) for each of the g ...
I'm using Spark 1.6 with scala; I have to compute the duration which is the difference between end time and start time. I've tried this: I want to ...
I'm having some difficulty combining several functions to do what I want in a 70000+ line excel file. ANY tips or pointers or advice greatly appreciat ...
i'm convering a pig script to spark 1.6 using scala, i have a dataframe which contains a string, and i want to swap characters in a certain order. exa ...
I have done this code on databricks environment but when I try it on my local env it breaks... Error: version is Spark 2.1 ...
I have created a UDF in BigQuery and managed to run it like the example in the documentation (https://cloud.google.com/bigquery/user-defined-functions ...
I have a udf stream with filter and map in Aerospike. If i map, as per all examples i have seen, i can pick fields from the record and return a new m ...
I am working on a Hive UDF in Scala I tried null.asInstanceOf[Double] but this gives the output as 0. I need a NULL output in hive instead. Thank ...
I have a JDBC connection to an Oracle DB. I also have some function f(x) written in Groovy or Scala. For example, f(x) simply returns 2x. Now my ques ...
I'm trying to create a new column in a DataFrame. This new column will contain a formatted data string created from a Long timestamp in milliseconds. ...
I am about to test the deterministic flag for SUDFs that return multiple values (follow up question to this). The DETERMINISTIC flag should cache the ...
This artcle gives a great overview on how to change columnnames. How to change dataframe column names in pyspark? Nontheless I need something more / ...
I'm trying to compute multiple values and fetch them in a select clause. Whether its computed via UDF or procedure does not matter to me but I can't f ...
Imagine the following code: How can I define the return type for myUdf so that people looking at the code will know immediately that it returns a D ...
I'm researching how to create a UDF to replicate a complete record of a Firebird table using triggers. I want to create a revision/history about som ...