create column based on conditions

Question

So my goal here is to see if the certain conditions apply to data I have

I have this query:

select colA, colB
from tableA
where colB in ('catA','catB')
AND
colA in ('value1', 'value2', ..., 'value999') 
or colA in ('value1000', ..., 'value1999') 
or colA in ('value1000', ..., 'value1999') 
or colA in ('value1000', ..., 'value1999') 

The reason I have all the OR clauses is because I have so much data and can only do 1000 items in a list at a time.

So im trying to solve two problems here.

A. The query above doesn't work...it says

ORA-00936: missing expression

It works with just the first part of the list so I know something is wrong with the OR clause.

B. How do I modify the select statement to get a new Yes/No column if this condition is met:

where colB in ('catA','catB')

I get a Yes, otherwise a No?

Answer 1

Even though this is a one-time query, this is a really bad idea. Oracle has the 1,000 "in list" limitation for really good reasons. Even a single in-list with 1,000 items takes a while just to compile. If you contrast this with a simple join, the compilation time is next to nothing, and the execution time is no more than it would be with your in list.

This is also a much more courteous approach to others on the database, as you are not hogging the shared pool with thousands of literals.

Even if this is a one-time thing, I'd recommend you create a table (or a GTT), load your values and do this as a simple join:

create table temporary_values (
  in_list_string varchar2(1000) not null,
  constraint temporary_values_pk primary key (in_list_string)
);

Use a tool like SQL*Loader to load your values, or do it the old-fashioned way. Then run your query as this:

select A.colA, A.colB
from
  tableA A
  join temporary_values t on
    A.colA = t.in_list_string
where
  A.colB in ('catA', 'catB');

If the primary key is a pain, you can always drop it and do your query as a semi-join to avoid duplicates:

select A.colA, A.colB
from
  tableA A
where
  A.colB in ('catA', 'catB') and
  exists (
    select null
    from temporary_values t
    where A.colA = t.in_list_string
  );

And if your and ever becomes an or you can resolve that by using a left join:

select A.colA, A.colB
from
  tableA A
  left join temporary_values t on
    A.colA = t.in_list_string
where
  t.in_list_string is not null or
  A.colB in ('catA', 'catB');

If there are any compilation or execution errors, they will be a lot easier to find than searching for a missing comma or misplaced quote in thousands of literals.