如何排除在同一 start_dt 上的数据中创建循环的记录？

Question

To identify cycle I can do select * from input A join input B on A.prv=B.cur and A.cur=B.prv but How to keep only one record in cycle having same start_dt? Except Prv and curr all columns are same for these records.I am using spark sql/Hive

Input

prv  cur  start_dt
A     B   2099-12-31
B     A   2099-12-31
P     Q   2018-12-31
Q     P   2018-12-31

Output (any of one record in cycle)
prv  cur  start_dt
A     B   2099-12-31
P     Q   2018-12-31

Answer 1

If you don't have Prv = Cur record (such as A, A, 2099-12-31 , then you could use:

    SELECT * FROM input A 
    JOIN input B ON A.prv=B.cur AND A.cur=B.prv
    WHERE A.prv > B.prv

Answer 2

Based on your input data, you can do:

select i.*
from input i
where i.prv < i.cur;

More generally, if you do not have duplicates for all pairs:

select i.*
from input i
where i.prv < i.cur
union all
select i.*
from input i
where i.prv > i.cur and
      not exists (select 1
                  from input i2
                  where i2.prv = i.cur and
                        i2.cur = i.prv and
                        i2.start_dt = i.start_dt 
                 );

Or, you can use row_number() :

select i.*
from (select i.*
             row_number() over (partition by start_dt, least(prv, cur), greatest(prv, cur) order by start_dt) as seqnum
      from input i
     ) i
where seqnum = 1;

This might be the most efficient method in Hive.