如何使用 Spark SQL 查询反转数组中的元素？

Question

I have a data like below Input data Key data a [5,2,6,null,null] b [5,7,9,4,null,null,null]

I want output to be like below. Output: Key data a [6,2,5,null,null] b [4,9,7,5,null,null,null]

Basically elements in the array needs to be reversed by keeping nulls at the end as it is. Can someone please help me with spark SQL query?

Answer 1

My approach - transform NULLs for sorting then transform back to NULL

select transform(sort_array(transform(data, x -> coalesce(x, 0)), False), x -> case when x=0 then null else x end) from table1

[EDIT]

Just noticed the transformations are not required if the NULLs are to be sorted at the end based on reverse order. sort_array() will work by itself

sort_array(data, False)

[EDIT 2]

Having it pointed out to me I misunderstood the question, I believe this will work... it's a little convoluted however:

select 
 concat( 
    reverse(array_except(array(5,7,9,4,null,null,null), array(null))) 
  , array_repeat(null
    , aggregate(
        transform(array(5,7,9,4,null,null,null), (x, i) -> (case when x is null then 1 else 0 end))
        , 0, (acc, x) -> acc + x
       )
     )
 )

The approach counts the number of nulls, removes the nulls, reverses the array and adds back the nulls at the end of the array

Answer 2

Untested:

reverse(filter(array(0, null, 2, 3, null), x -> x IS NOT NULL))

then append:

filter(array(0, null, 2, 3, null), x -> x IS NULL)

See Filter