在Vertica中查找集合的交集

Question

I'm trying to sort out how to find the intersection of a pair of sets in Vertica (and a better way to do it).

I have 2 sets the first set is larger and stored in a single column table:

San Francisco
New York
Chicago
London
Rome

The second set is stored as a delimited string in a Varchar field and can include items not in set 1 each set is a single string entry per row

San Francisco,Chicago,Tampa
Tampa,New Orleans,Miami

What I need to be able to do is efficiently choose the members of the second set that are in the first set and then get the intersection of the two sets for further processing so for the second set I need it to return:

{San Francisco,Chicago}
{}

So I need to be able to go through the table containing the set 2s and get that information then do something else for each instance of an intersection with the row that contains it.

Suggestions, please!

Answer 1

I get as far as generating a tabular output containing what you are looking for. Exporting that in JSON format, as you seem to expect, is, in my eyes, the front end's job, not the database's.

Having said that, see here:

-- input 1: one city per column
WITH city(city) AS (
          SELECT 'San Francisco'
UNION ALL SELECT 'New York'
UNION ALL SELECT 'Chicago'
UNION ALL SELECT 'London'
UNION ALL SELECT 'Rome'
)
,
-- input 2: many cities per column
cities(cities) AS (
          SELECT 'San Francisco,Chicago,Tampa'
UNION ALL SELECT 'Tampa,New Orleans,Miami'
)
,
-- end of input. Start "real" WITH clause here.
i(i) AS ( -- index for SPLIT_PART()
          SELECT  1 
UNION ALL SELECT  2 
UNION ALL SELECT  3 
UNION ALL SELECT  4 
UNION ALL SELECT  5 
UNION ALL SELECT  6 
UNION ALL SELECT  7 
UNION ALL SELECT  8 
UNION ALL SELECT  9 
UNION ALL SELECT 10
)
,
-- verticalise all those side-by-side cities, using SPLIT_PART() 
-- and the index table above
pivot_cities AS (
  SELECT DISTINCT
    SPLIT_PART(cities,',',i) AS city
  FROM cities CROSS JOIN i
)
-- INNER JOIN input 1 with the distinct verticalised cities of input 2
SELECT
  city.city
FROM city
JOIN pivot_cities USING(city)
;

-- result:
city
-------------
Chicago
San Francisco

Answer 2

Here is a way of doing it that doesn't require a manual pivot using UNION ALL and an assumed number of maximum items in the list. In this example, the table t_city is the the one with single-entries in the column and t_cities is the one with multiple entries in the column:

WITH cte_cities AS (
    SELECT id, v_txtindex.StringTokenizerDelim(cities, ',') OVER (PARTITION BY id)
      FROM t_cities
)
    SELECT cte_cities.id AS cities_id,
           cte_cities.words AS city 
      FROM cte_cities
INNER JOIN t_city ON t_city.city = cte_cities.words
  GROUP BY cte_cities.id,
           cte_cities.words
  ORDER BY cte_cities.id

This will return rows with the id, which will allow you to aggregate them on the front-end of your application. If you require that they are re-assembled back into a comma delimited list then you will need to install the Vertica Strings Extension Package and use the function group_concat which should give you the results you are looking for.