Extract last N elements of an array in SQL (hive)

Question

I have a column with arrays and I want to extract the X last elements in an array.

Example trying to extract the last two elements:

     Column A
     ['a', 'b', 'c']
     ['d', 'e']
     ['f', 'g', 'h', 'i']

Expected output:

     Column A
    ['b', 'c']
    ['d', 'e']
    ['h', 'i']

Best case scenario would be to do it without using a UDF

Answer 1

One method using reverse, explode, filtering and re-assembling array again:

with your_table as (
select stack (4,
0, array(), --empty array to check it works if no elements or less than n
1, array('a', 'b', 'c'),
2, array('d', 'e'),
3, array('f', 'g', 'h', 'i')
) as (id, col_A)
)

select s.id, collect_list(s.value) as col_A 
from
(select s.id, a.value, a.pos
  from your_table s
       lateral view outer posexplode(split(reverse(concat_ws(',',s.col_A)),',')) a as pos, value
where a.pos between 0 and 1 --last two (use n-1 instead of 1 if you want last n)  
distribute by s.id sort by a.pos desc --keep original order
)s
group by s.id

Result:

s.id    col_a
0   []
1   ["b","c"]
2   ["d","e"]
3   ["h","i"]

More elegant way using brickhouse numeric_range UDF in this answer