简体   繁体   中英

Writing unit test for a spark scala function that takes no arguments?

I am writing unit tests of scala function where I am passing mocked spark data frame to the function and then using assertSmallDataFrameEquality(actualDF, expectedDF) function to check whether my function is transforming correctly or not.

Recently I came across a function that is taking no argument and returning Column type. Now since it is not expecting any argument. How should I write test case for this function. My function is given below.

def arriveDateMinusRuleDays: Column = {
  expr(s"date_sub(${Columns.ARRIVE_DATE},${Columns.RULE_DAYS})")
}

Test blueprint is written below

test("arrive date minus rule days") {
  import spark.implicits._
  val today = Date.valueOf(LocalDate.now)

  val inputDF = Seq(
    (Y, today, 0, 80852),
    (S, today, 1, 18851))
    .toDF(FLAG, ARRIVE_DT, RULE_DAYS,ITEM_NBR)

  val actualOutput = DataAggJob.arriveDateMinusRuleDays()   // How to pass my column values to this function
// val exepectedoutput
 assertmethod(actualoutput, expectedoutput)
//  print(actualOutput)
}

You don't need to test each individual function. The purpose of the unit test is to assert the contract between implementation and downstream consumer, not implementation details.

If your job returns the expected output given the input, then it is working correctly, regardless of what that particular function is doing. It should really just be made private to avoid confusion,

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM