Show duplicates in internal table

Question

Each an every item should have an uniquie SecondNo + Drawing combination. Due to misentries, some combinations are there two times.

I need to create a report with ABAP which identifies those combinations and does not reflect the others.

Item:  SecNo: Drawing:
121       904      5000         double
122       904      5000         double
123       816      5100
124       813      5200
125       812      4900          double
126       812      4900          double
127       814      5300

How can I solve this? I tried 2 approaches and failed:

1. Sorting the data and tried to print out each one when the value of the upper row is equal to the next value

2. counting the duplicates and showing all of them which are more then one.

Where do I put in the condition? in the loop area?

I tried this:

REPORT  duplicates.

DATA: BEGIN OF lt_duplicates OCCURS 0,
        f2(10),
        f3(10),
      END OF lt_duplicates,
      it_test TYPE TABLE OF ztest WITH HEADER LINE,
      i       TYPE i.

SELECT DISTINCT f2 f3 FROM ztest INTO TABLE lt_duplicates.

LOOP AT lt_duplicates.

  IF f2 = lt_duplicates-f2 AND f3 = lt_duplicates-f3.
  ENDIF.

  i = LINES( it_test ).

  IF i > 1.
    LOOP AT it_test.
      WRITE :/ it_test-f1,it_test-f2,it_test-f3.
    ENDLOOP.
  ENDIF.

ENDLOOP.

Answer 1

You can use AT...ENDAT for this, provided that you arrange the fields correctly:

TYPES: BEGIN OF t_my_line,
         secno   TYPE foo,
         drawing TYPE bar,
         item    TYPE baz, " this field has to appear AFTER the other ones in the table
       END OF t_my_line.

DATA: lt_my_table TYPE TABLE OF t_my_line,
      lt_duplicates TYPE TABLE OF t_my_line.

FIELD-SYMBOLS: <ls_line> TYPE t_my_line.

START-OF-WHATEVER.

* ... fill the table ...

  SORT lt_my_table BY secno drawing.
  LOOP AT lt_my_table ASSIGNING <ls_line>.
    AT NEW drawing. " whenever drawing or any field left of it changes...
      FREE lt_duplicates.
    ENDAT.
    APPEND <ls_line> TO lt_duplicates.
    AT END OF drawing.
      IF lines( lt_duplicates ) > 1.
*       congrats, here are your duplicates...
      ENDIF.
    ENDAT.
  ENDLOOP.

Answer 2

From ABAP 7.40, you may use the GROUP BY constructs with the GROUP SIZE words so that to take into account only the groups with at least 2 elements.

• ABAP statement LOOP AT ... GROUP BY ( <columns...> gs = GROUP SIZE ) ...
  • Loop at grouped lines:
    1. Either LOOP AT GROUP ...
    2. Or ... FOR ... IN GROUP ...
• ABAP expression ... VALUE|REDUCE|NEW type|#( FOR GROUPS ... GROUP BY ( <columns...> gs = GROUP SIZE ) ...
  • Loop at grouped lines: ... FOR ... IN GROUP ...

For both constructs, it's possible to loop at the grouped lines in two ways: * LOOP AT GROUP ... * ... FOR ... IN GROUP ...

Line#  Item    SecNo   Drawing
  1    121       904      5000         double
  2    122       904      5000         double
  3    123       816      5100
  4    124       813      5200
  5    125       812      4900         double
  6    126       812      4900         double
  7    127       814      5300

You might want to produce the following table containing the duplicates:

SecNo   Drawing   Lines
904      5000     [1,2]
812      4900     [5,6]

Solution with LOOP AT ... GROUP BY ... :

TYPES: BEGIN OF t_line,
         item    TYPE i,
         secno   TYPE i,
         drawing TYPE i,
       END OF t_line,
       BEGIN OF t_duplicate,
         secno   TYPE i,
         drawing TYPE i,
         num_dup TYPE i, " number of duplicates
         lines   TYPE STANDARD TABLE OF REF TO t_line WITH EMPTY KEY,
       END OF t_duplicate,
       t_lines      TYPE STANDARD TABLE OF t_line WITH EMPTY KEY,
       t_duplicates TYPE STANDARD TABLE OF t_duplicate WITH EMPTY KEY.

DATA(table) = VALUE t_lines(
    ( item = 121 secno = 904 drawing = 5000 )
    ( item = 122 secno = 904 drawing = 5000 )
    ( item = 123 secno = 816 drawing = 5100 )
    ( item = 124 secno = 813 drawing = 5200 )
    ( item = 125 secno = 812 drawing = 4900 )
    ( item = 126 secno = 812 drawing = 4900 )
    ( item = 127 secno = 814 drawing = 5300 ) ).

DATA(expected_duplicates) = VALUE t_duplicates(
    ( secno = 904 drawing = 5000 num_dup = 2 lines = VALUE #( ( REF #( table[ 1 ] ) ) ( REF #( table[ 2 ] ) ) ) )
    ( secno = 812 drawing = 4900 num_dup = 2 lines = VALUE #( ( REF #( table[ 5 ] ) ) ( REF #( table[ 6 ] ) ) ) ) ).

DATA(actual_duplicates) = VALUE t_duplicates( ).
LOOP AT table
    ASSIGNING FIELD-SYMBOL(<line>)
    GROUP BY
    ( secno   = <line>-secno
      drawing = <line>-drawing
      gs      = GROUP SIZE )
    ASSIGNING FIELD-SYMBOL(<group_table>).

  IF <group_table>-gs >= 2.
    actual_duplicates = VALUE #( BASE actual_duplicates
        ( secno   = <group_table>-secno
          drawing = <group_table>-drawing
          num_dup = <group_table>-gs
          lines   = VALUE #( FOR <line2> IN GROUP <group_table> ( REF #( <line2> ) ) ) ) ).
  ENDIF.

ENDLOOP.

WRITE : / 'List of duplicates:'.
SKIP 1.
WRITE : / 'Secno       Drawing     List of concerned items'.
WRITE : / '----------  ----------  ---------------------------------- ...'.
LOOP AT actual_duplicates ASSIGNING FIELD-SYMBOL(<duplicate>).
  WRITE : / <duplicate>-secno, <duplicate>-drawing NO-GROUPING.
  LOOP AT <duplicate>-lines INTO DATA(line).
    WRITE line->*-item.
  ENDLOOP.
ENDLOOP.

ASSERT actual_duplicates = expected_duplicates. " short dump if not equal

Output:

List of duplicates:

Secno       Drawing     List of concerned items
----------  ----------  ---------------------------------- ...
       904        5000         121         122
       812        4900         125         126

Solution with ... VALUE type|#( FOR GROUPS ... GROUP BY ... :

DATA(actual_duplicates) = VALUE t_duplicates(
    FOR GROUPS <group_table> OF <line> IN table
    GROUP BY
    ( secno   = <line>-secno
      drawing = <line>-drawing
      gs      = GROUP SIZE )
    ( secno   = <group_table>-secno
      drawing = <group_table>-drawing
      num_dup = <group_table>-gs
      lines   = VALUE #( FOR <line2> IN GROUP <group_table> ( REF #( <line2> ) ) ) ) ).
DELETE actual_duplicates WHERE num_dup = 1.

Note: for deleting non-duplicates, instead of using an additional DELETE statement, it can be done inside the VALUE construct by adding a LINES OF COND construct which adds 1 line if group size >= 2, or none otherwise (if group size = 1):

      ...
      gs      = GROUP SIZE )
    ( LINES OF COND #( WHEN <group_table>-gs >= 2 THEN VALUE #( "<== new line
    ( secno   = <group_table>-secno
      ...
      ... REF #( <line2> ) ) ) ) ) ) ) ). "<== 3 extra right parentheses

Answer 3

I needed simply to report duplicate lines in error based on two fields so used the following.

LOOP AT gt_data INTO DATA(gs_data)

GROUP BY ( columnA = gs_data-columnA columnB = gs_data-columnB
size = GROUP SIZE index = GROUP INDEX ) ASCENDING
REFERENCE INTO DATA(group_ref).

IF group_ref->size > 1.
  PERFORM insert_error USING group_ref->columnA group_ref->columnB.
ENDIF.

ENDLOOP.

Answer 4

Here is my 2p worth, you could cut some out of this depending on what you want to do, and you should consider the amount of data being processed too. This method is only really for smaller sets. Personally I like to prevent erroneous records at the source. Catching an error during input. But if you do end up in a pickle there is definitely more than one way to solve the issue.

TYPES: BEGIN OF ty_itab,
         item    TYPE i,
         secno   TYPE i,
         drawing TYPE i,
       END OF ty_itab.
TYPES: itab_tt TYPE STANDARD TABLE OF ty_itab.

DATA: lt_itab  TYPE itab_tt,
      lt_itab2 TYPE itab_tt,
      lt_itab3 TYPE itab_tt.

lt_itab = VALUE #(
                  ( item = '121' secno = '904' drawing = '5000' )
                  ( item = '122' secno = '904' drawing = '5000' )
                  ( item = '123' secno = '816' drawing = '5100' )
                  ( item = '124' secno = '813' drawing = '5200' )
                  ( item = '125' secno = '812' drawing = '4900' )
                  ( item = '126' secno = '812' drawing = '4900' )
                  ( item = '127' secno = '814' drawing = '5300' )
                ).

APPEND LINES OF lt_itab TO lt_itab2.
APPEND LINES OF lt_itab TO lt_itab3.

SORT lt_itab2 BY secno drawing.
DELETE ADJACENT DUPLICATES FROM lt_itab2 COMPARING secno drawing.   

* Loop at what is hopefully the smaller itab.
LOOP AT lt_itab2 ASSIGNING FIELD-SYMBOL(<line>).
  DELETE TABLE lt_itab3 FROM <line>.
ENDLOOP.

* itab1 has all originals.
* itab2 has the unique.
* itab3 has the duplicates.