在SQL Server中查找关联

Question

Just want to know if the following can be done ENTIRELY in SQL Server

I have a table which has 3 columns - SENTENCE ID (PK) , SENTENCE (strings of arbitrary length), PATTERNS (these are 2 or 3 word patterns which are found in the SENTENCE ).

I need to find the correlation of all the distinct PATTERNS with each other.

If I do it externally (using python and ODBC) I need to go through the following steps

FOR each distinct PATTERN

1. Get the count of PATTERN
2. Find all the sentence IDs that have that PATTERN
3. Get counts of all PATTERNS that occur in the above sentence IDs
4. Append the current PATTERN and its count (as columns) to the table in step3.
5. keep appending the above table as rows to the result table

Next

Answer 1

Let me assume that PATTERN follows the form of a like expression. And, that you want to count a pattern for a sentence only once.

If so, you can do the following. Get the matches between all sentences and patterns:

with sp as (
       select s.sentenceID, p.pattern, count(*) over (partition by p.pattern) as NumSentences
       from Sentences s join
            Patterns p
            on s.sentence like p.pattern
     )
select sp1.pattern, sp2.pattern,
       sp1.pattern as Pattern1Count, sp2.pattern as Pattern2Count,
       count(*) as BothCount
from sp sp1 join
     sp sp2
     on sp1.pattern < sp2.pattern    -- <= if you want counts for a single pattern
group by sp1.pattern, sp2.pattern

You don't explicitly say what kind of output you want, but this should be sufficient.

So, with some reasonable assumptions, you can do this in SQL.