Scala：如何在循环中合并数据帧

Question

I would like to do union of data frame in the recursive method.

I am doing some calculations in the recursive method and filtering the data and storing in one variable. In 2nd iteration i will do some calculation and again i will store the data in same variable.when i am calling the method second time my first result is getting vanished.Ideally i have to store the result in one temp variable and i need to do union of all the result till the recursive method gets completed its execution.

Iteration1 output in df:

Col1   
    14      
    35    

Iteration2 output in df:

Col1
    18      
    20

Now i need the final output as,

Col1
    14
    35
    18
    20

Code:

def myRecursiveMethod(first: List[List[String]],
                        Inputcolumnsdummy: List[List[String]],
                        secondInputcolumns: List[List[String]] = {

  val ongoingResult = doSomeCalculation(first,Inputcolumnsdummy, secondInputcolumns)
}

I want my code should be something like below,

def myRecursiveMethod(first: List[List[String]],
                        Inputcolumnsdummy: List[List[String]],
                        secondInputcolumns: List[List[String]]) = {

    val ongoingResult = doSomeCalculation(first, Inputcolumnsdummy, secondInputcolumns)
    Val temp = temp.union(ongoingResult)
}

Answer 1

You should try: use union like this: df1.union(df2) or df1.union(computation(df2,...)) .

Exemple below:

def doCompute(df: DataFrame): DataFrame = {
    val tmp: DataFrame = ... // TODO: call to your computation method
    tmp.show()
    df.union(tmp)
}

val df1: DataFrame = ...
val df2: DataFrame = ...
val df3: DataFrame = ...

var union_df: DataFrame = df1.union(doCompute(df2)).union(doCompute(df3))

One thing I did not understand in your question is how is your function myRecursiveMethod recursive? A recursive function calls itself, by definition. Not sure your question is really clear.