从内部表填充范围表，没有重复项

Question

I wonder what's usually faster:

• Filter out duplicates and then do the select
  or
• Do the select directly with duplicates

I think it may be the first one but I don't know -
how to nicely and efficiently integrate the deletion of duplicates in my code?

DATA:
  lt_itab       TYPE TABLE OF string,
  lt_range_itab TYPE RANGE OF string
  .

* Populating itab with duplicates
* Can the following somehow become a neat one-liner? This is ugly!
APPEND '1'  TO lt_itab.
APPEND '2'  TO lt_itab.
APPEND '2'  TO lt_itab.
APPEND '3'  TO lt_itab.
APPEND '4'  TO lt_itab.
APPEND '4'  TO lt_itab.

*Populating range table from itab
*Should one remove the duplicates here for a performance boost in the upcoming select?
*If so - how?
*-------------------------------------------------------
lt_range_itab = VALUE #(
  FOR <ls_itab> IN lt_itab
  ( sign = 'I'
    option = 'EQ'
    low = <ls_itab> )
).

*...or is such a select usually faster than the time it takes to remove the duplicates?
*-------------------------------------------------------
*SELECT *
*  FROM       anyTable
*  INTO TABLE @DATA(lt_new_data)
*  WHERE      anyProperty NOT IN @lt_range_itab.

Answer 1

So many questions... :-)

Edit: I revised this answer after the discussions in the answers

What's usually faster?

Good databases will select fast even if there are some duplicates in the range. Oracle's optimizer, for example, removes duplicates on its own . SAP HANA, in comparison, may get slower, but its dictionary-based architecture will usually keep it on a negligible level. So generally, I see no imperative to remove duplicates before each and every query.

However, things may go awry if the optimizer is sub-optimal and there is a large number of duplicates. So if you expect duplicates, it may be better to keep to the safe side and remove them before the query.

Also note that range tables have a length restriction. They are translated to a SQL statement with an IN clause, and SQL statement strings have a maximum number of characters. Duplicate removal thus may be a necessary strategy to get the query working at all.

Longer ranges can be converted to FOR ALL ENTRIES , which packetizes the query and allows for much longer ranges. However, that statement form leads to multiple roundtrips with the database, and will definitely suffer from duplicates in the query.

Can the following somehow become a neat one-liner?

DATA(lt_itab) = VALUE string_table( ( `1` ) ( `2` ) ( `2` ) ( `3` ) ( `4` ) ( `4` ) ).

Or right away as suggested by Sandra below:

SELECT ... WHERE anyProperty NOT IN ('1','2','3','4')

If so - how?

SORT lt_range_tab.
DELETE ADJACENT DUPLICATES FROM lt_range_tab.

Last but not least, note that an anyProperty IN @lt_range_tab may be considerably faster than the reversed NOT IN variant. Databases tend to keep positive indexes which respond best to positive queries. If you have the possibility, eg because you're filtering a field with fixed value list, it may be worthwhile to reverse the filter before sending it to the database.

Answer 2

The first one is faster of course, because the database operations with physical disks are much slower than in-memory operations.

Is the impact on performance noticeable is another question ; it depends on the number of duplicate selections and on the volume of data.

One well known example is the SELECT ... FOR ALL ENTRIES construct which can have a big impact on performance if the duplicates are not removed, because ABAP internally converts it into several SELECT, and so the same data may be selected several times (which is removed at ABAP side afterwards).

In brief, except when you are sure there is a small data volume, make sure there are no duplicates before a SELECT.