Moving average in Apache spark dataframe C#

Question

have some samples in my db I have imported into apache spark Dataframe. I need to add another column with moving average of n samples before current sample in C#.

This is my dataframe:

DataFrame frame = jdbcDf
            .Where("SubjectKey = 104")
            .Select("Timestamp", "Current").Sort("Timestamp");

I have found this code:

 val movAvg = sampleData.withColumn("movingAverage", AVG(sampleData("Current"))
             .over( Window.partitionBy("Role").rowsBetween(-1,1)) )

But To use window class in my code. How do I calculate moving averege? How do I handle Null values while doing so?

Answer 1

I have manage to find to correct syntax:

DataFrame frame = jdbcDf
            .Where("SubjectKey = 104")
            .Select("Timestamp", "Current");
var windowSp = Microsoft.Spark.Sql.Expressions.Window.RowsBetween(-10, -1);
var movAvg = result.Sort("Timestamp").WithColumn("movingAverage", 
Avg(result["Diff"]).Over(windowSp));