SQL Python中的存儲過程

Question

我正在嘗試編寫 Python 代碼來模仿 SQL 動態存儲的產品。 我只是在乞討。 我已使用 pyodbc 成功連接到 SQL DB 並將數據輸入 Pandas df。 1.我已經成功地從Survey表中獲取所有SurveyId來構建Python中的所有SurveyId列表。 這是調查表： 使用此代碼，我從調查表中獲取所有 SurveyId 列表：

def surveyCursor():
    query_GetSurveyId = 'SELECT SurveyId FROM Survey'
    cursor = sql_conn.cursor()
    cursor.execute(query_GetSurveyId)
    row = [item[0] for item in cursor.fetchall()]
    return row

surveyId_list = surveyCursor()
#print(surveyId_list)

2. 現在有了下一個 function，我已經有了這個 SQL 存儲過程，如下所示：

DECLARE currentQuestionCursor CURSOR FOR
            SELECT *
                    FROM
                    (
                        SELECT
                            SurveyId,
                            QuestionId,
                            1 as InSurvey
                        FROM
                            SurveyStructure
                        WHERE
                            SurveyId = @currentSurveyId
                        UNION
                        SELECT 
                            @currentSurveyId as SurveyId,
                            Q.QuestionId,
                            0 as InSurvey
                        FROM
                            Question as Q
                        WHERE NOT EXISTS
                        (
                            SELECT *
                            FROM SurveyStructure as S
                            WHERE S.SurveyId = @currentSurveyId AND S.QuestionId = Q.QuestionId
                        )
                    ) as t
                    ORDER BY QuestionId;

我需要在 Python 中使用它，因此我創建了這個 function：

def CheckInSurvey(SurveyList):
    for surveyId in SurveyList:
        if surveyId < len(SurveyList):
            query_QuestionInSurvey = 'SELECT * FROM (SELECT SurveyId, QuestionId, 1 as InSurvey \
                        FROM SurveyStructure WHERE SurveyId = ' + str(surveyId) + \
                        ' UNION SELECT ' + str(surveyId) + ' as SurveyId, Q.QuestionId, 0 as InSurvey \
                        FROM Question as Q WHERE NOT EXISTS ( SELECT * FROM SurveyStructure as S \
                        WHERE S.SurveyId = ' + str(surveyId) + ' AND S.QuestionId = Q.QuestionId)) as t UNION '
            cursor = sql_conn.cursor()
            cursor.execute(query_QuestionInSurvey)
            row = [item[0] for item in cursor.fetchall()]
            #checkQuestionInSurvey = query_QuestionInSurvey
        else:
            query_QuestionInSurvey_LastRow = 'SELECT * FROM (SELECT SurveyId, QuestionId, 1 as InSurvey \
                        FROM SurveyStructure WHERE SurveyId = ' + str(surveyId) + \
                        ' UNION SELECT ' + str(surveyId) + ' as SurveyId, Q.QuestionId, 0 as InSurvey \
                        FROM Question as Q WHERE NOT EXISTS ( SELECT * FROM SurveyStructure as S \
                        WHERE S.SurveyId = ' + str(surveyId) + ' AND S.QuestionId = Q.QuestionId)) as t ORDER BY QuestionId '
            cursor = sql_conn.cursor()
            cursor.execute(query_QuestionInSurvey_LastRow)
            row = [item[0] for item in cursor.fetchall()]
            #checkQuestionInSurvey = query_QuestionInSurvey_LastRow
        surveyId += 1
    return row

InSurveyList = CheckInSurvey(surveyId_list)

對於這個 function，我想檢查當前 QuestionId 以與調查結構表進行比較： 如果當前 QuestionId 在調查結構表中，則將 InSurvey 設置為 1，否則將 InSurvey 設置為 0。

我想得到一張這樣的表來放入 dataframe：

但是，當我編譯我的代碼時，我得到了這個錯誤：

你能告訴我這里可能是什么問題嗎？ 非常感謝您提前。

這是整個 Python 代碼：

import pandas as pd
import pyodbc as odbc

# Make a connection to database using pyodbc, then store the query in a variable then put it in pandas dataframe
# To check which ODBC Driver to use:
# 1. From Start menu, go to ODBC Data Sources
# 2. Clieck the Drivers tab, then find SQL Server ODBC Driver in the list of ODBC drivers that installed on the system
sql_conn = odbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER=LAPTOP-NAD8U5G4;DATABASE=Survey_Sample_A19;Trusted_Connection=yes')
query = "SELECT * FROM [Answer]"
sql_to_df = pd.read_sql(query, sql_conn)

#print(sql_to_df.head(3))

# Section 1: Stored Procedue
# Inner loop:
# 1. Iterate in the Survey table to get all SurveyId
# 2. After we get all SurveyId, this will become @currentSurveyId to be used in the loop below in step 3
# Iterate through each Question using "currentQuestionCursor"
# 3. Check if current QuestionId is in SurveyStructure table:
# Yes -> set InSurvey = 1
#     Then, get current QuestionId from Answer table (also SurveyId from Answer table?????), then update Answer string "strQueryTemplateForAnswerColumn", replace with 
#     strColumnsQueryPart AS ANS_Q(currentQuestionID)
# No -> set InSurvey = 0
#     Then, update Answer string "strQueryTemplateForNullColumnn" to
#     NULL AS ANS_Q(currentQuestionID)
# 4. Select SurveyId, AnswerId, then update Answer string
# 5. After the last row, concatenate Answer string "strQueryTemplateForAnswerColumn" with ","
# 6. Remove the cursor "currentQuestionCursor" when finish the Inner

def surveyCursor():
    query_GetSurveyId = 'SELECT SurveyId FROM Survey'
    cursor = sql_conn.cursor()
    cursor.execute(query_GetSurveyId)
    row = [item[0] for item in cursor.fetchall()]
    return row

surveyId_list = surveyCursor()
#print(surveyId_list)

def CheckInSurvey(SurveyList):
    for surveyId in SurveyList:
        if surveyId < len(SurveyList):
            query_QuestionInSurvey = 'SELECT * FROM (SELECT SurveyId, QuestionId, 1 as InSurvey \
                        FROM SurveyStructure WHERE SurveyId = ' + str(surveyId) + \
                        ' UNION SELECT ' + str(surveyId) + ' as SurveyId, Q.QuestionId, 0 as InSurvey \
                        FROM Question as Q WHERE NOT EXISTS ( SELECT * FROM SurveyStructure as S \
                        WHERE S.SurveyId = ' + str(surveyId) + ' AND S.QuestionId = Q.QuestionId)) as t UNION '
            cursor = sql_conn.cursor()
            cursor.execute(query_QuestionInSurvey)
            row = [item[0] for item in cursor.fetchall()]
            #checkQuestionInSurvey = query_QuestionInSurvey
        else:
            query_QuestionInSurvey_LastRow = 'SELECT * FROM (SELECT SurveyId, QuestionId, 1 as InSurvey \
                        FROM SurveyStructure WHERE SurveyId = ' + str(surveyId) + \
                        ' UNION SELECT ' + str(surveyId) + ' as SurveyId, Q.QuestionId, 0 as InSurvey \
                        FROM Question as Q WHERE NOT EXISTS ( SELECT * FROM SurveyStructure as S \
                        WHERE S.SurveyId = ' + str(surveyId) + ' AND S.QuestionId = Q.QuestionId)) as t ORDER BY QuestionId '
            cursor = sql_conn.cursor()
            cursor.execute(query_QuestionInSurvey_LastRow)
            row = [item[0] for item in cursor.fetchall()]
            #checkQuestionInSurvey = query_QuestionInSurvey_LastRow
        surveyId += 1
    return row

InSurveyList = CheckInSurvey(surveyId_list)

並且，SQL 存儲過程：

USE [Survey_Sample_A18]
GO
/****** Object:  UserDefinedFunction [dbo].[fn_GetAllSurveyDataSQL]    Script Date: 2/8/2020 4:31:13 PM ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
-- =============================================
-- Author:      <Author,,Name>
-- Create date: <Create Date, ,>
-- Description: <Description, ,>
-- =============================================
CREATE OR ALTER FUNCTION [dbo].[fn_GetAllSurveyDataSQL]() 
RETURNS nvarchar(max)
AS
BEGIN

    DECLARE @strQueryTemplateForAnswerColumn nvarchar(max);
    DECLARE @strQueryTemplateForNullColumnn nvarchar(max);
    DECLARE @strQueryTemplateOuterUnionQuery nvarchar(max);
    DECLARE @currentSurveyId int;

    -- WHEN YOU WRITE DYNAMIC SQL IN STRING VARIABLES
    -- IT'S LIKELY THAT YOU WILL CONCATENATE STRINGS LATER ON IN THE PROCESS
    -- SO, BE CAREFULL OF LEAVING SPACES BEFORE AND AFTER THE QUERY TEXT
    SET @strQueryTemplateForAnswerColumn = '
            COALESCE(
                (
                    SELECT a.Answer_Value
                    FROM Answer as a
                    WHERE
                        a.UserId = u.UserId
                        AND a.SurveyId = <SURVEY_ID>
                        AND a.QuestionId = <QUESTION_ID>
                ), -1) AS ANS_Q<QUESTION_ID> ';

    SET @strQueryTemplateForNullColumnn = ' NULL AS ANS_Q<QUESTION_ID> '

    SET @strQueryTemplateOuterUnionQuery = '
            SELECT
                    UserId
                    , <SURVEY_ID> as SurveyId
                    , <DYNAMIC_QUESTION_ANSWERS>
            FROM
                [User] as u
            WHERE EXISTS
            (
                    SELECT *
                    FROM Answer as a
                    WHERE u.UserId = a.UserId
                    AND a.SurveyId = <SURVEY_ID>
            )
    ';

    DECLARE @strCurrentUnionQueryBlock nvarchar(max);
    SET @strCurrentUnionQueryBlock = ''


    DECLARE @strFinalQuery nvarchar(max);
    SET @strFinalQuery = ''

    --Cursor variables are the only ones to not have an @ in front of their names
    DECLARE surveyCursor CURSOR FOR
                            SELECT SurveyId
                            FROM Survey
                            ORDER BY SurveyId;

    OPEN surveyCursor; -- when opened, the cursor is before the first row
    FETCH NEXT FROM surveyCursor INTO @currentSurveyId; --first row read

    WHILE @@FETCH_STATUS = 0 --AS LONG AS FETCH_STATUS IS EQUAL TO 0 --> there's a row to read
    BEGIN

        -- MAIN LOOP, OVER ALL THE SURVEYS

        -- FOR EACH SURVEY, IN @currentSurveyId, WE NEED TO CONSTRUCT THE ANSWER COLUMN QUERIES

        -- Another iteration, over the questions of the survey
        -- inner loop, another cursor

        --I want a resultset of SurveyId, QuestionId, flag InSurvey indicating whether
        -- the question is in the survey structure
        DECLARE currentQuestionCursor CURSOR FOR
            SELECT *
                    FROM
                    (
                        SELECT
                            SurveyId,
                            QuestionId,
                            1 as InSurvey
                        FROM
                            SurveyStructure
                        WHERE
                            SurveyId = @currentSurveyId
                        UNION
                        SELECT 
                            @currentSurveyId as SurveyId,
                            Q.QuestionId,
                            0 as InSurvey
                        FROM
                            Question as Q
                        WHERE NOT EXISTS
                        (
                            SELECT *
                            FROM SurveyStructure as S
                            WHERE S.SurveyId = @currentSurveyId AND S.QuestionId = Q.QuestionId
                        )
                    ) as t
                    ORDER BY QuestionId;

        DECLARE @currentSurveyIdInQuestion int;
        DECLARE @currentQuestionID int;
        DECLARE @currentInSurvey int;

        OPEN currentQuestionCursor;
        --When fetching into local variable, the column order of the select clause must be followed
        FETCH NEXT FROM currentQuestionCursor INTO @currentSurveyIdInQuestion,
                                                    @currentQuestionID,
                                                    @currentInSurvey;

        DECLARE @strColumnsQueryPart nvarchar(max);

        SET @strColumnsQueryPart = '';
        WHILE @@FETCH_STATUS = 0
        BEGIN
            --the "global" variable @@FETCH_STATUS GETS LOCALISED BETWEEN BEGIN --- END BLOCK

            --INNER LOOP IETERATES OVER THE QUESTION

            --IS THE CURRENT QUESTION (inner loop) IN THE CURRENT SURVEY (outer loop)

            IF @currentInSurvey = 0 -- CURRENT QUESTION IS NOT IN THE CURRENT SURVEY
            BEGIN
                --THEN BLOCK
                -- SPECIFICATION: THE VALUES IN THIS COLUMN WILL BE NULL
                SET @strColumnsQueryPart = @strColumnsQueryPart +
                        REPLACE(@strQueryTemplateForNullColumnn, '<QUESTION_ID>',
                                @currentQuestionID);
            END
            ELSE
            BEGIN
                SET @strColumnsQueryPart = @strColumnsQueryPart +
                    REPLACE(@strQueryTemplateForAnswerColumn, '<QUESTION_ID>',
                            @currentQuestionID);
            END;

            FETCH NEXT FROM currentQuestionCursor INTO @currentSurveyIdInQuestion,
                                                    @currentQuestionID,
                                                    @currentInSurvey;

        IF @@FETCH_STATUS = 0
        BEGIN
            -- Place a comma between column statements, except for the last one
            SET @strColumnsQueryPart = @strColumnsQueryPart + ' , ';
        END;

        END; -- END OF CLOSE INNER LOOP WHILE
        CLOSE currentQuestionCursor;
        DEALLOCATE currentQuestionCursor;


        --BACK IN THE OUTER LOOP OVER SURVEYS
        SET @strCurrentUnionQueryBlock = 
            REPLACE(@strQueryTemplateOuterUnionQuery,
                    '<DYNAMIC_QUESTION_ANSWERS>',
                    @strColumnsQueryPart);

        SET @strCurrentUnionQueryBlock = 
            REPLACE(@strCurrentUnionQueryBlock, 
                        '<SURVEY_ID>', @currentSurveyId);

        SET @strFinalQuery = @strFinalQuery + @strCurrentUnionQueryBlock

        FETCH NEXT FROM surveyCursor INTO @currentSurveyId;

        IF @@FETCH_STATUS = 0 
        BEGIN
            SET @strFinalQuery = @strFinalQuery + ' UNION ' ;
        END;

    END;

    CLOSE surveyCursor;
    DEALLOCATE surveyCursor;

    --calling the system store procedure sp_executesql
    --recommended by Microsoft should the text of your dunamic query > 4000 chars
     RETURN @strFinalQuery;


END

Answer 1

在CheckInSurvey中， query_QuestionInSurvey中的最后一個UNION可能會導致問題。

def CheckInSurvey(SurveyList):
for surveyId in SurveyList:
    if surveyId < len(SurveyList):
        query_QuestionInSurvey = 'SELECT * FROM (SELECT SurveyId, QuestionId, 1 as InSurvey \
                    FROM SurveyStructure WHERE SurveyId = ' + str(surveyId) + \
                    ' UNION SELECT ' + str(surveyId) + ' as SurveyId, Q.QuestionId, 0 as InSurvey \
                    FROM Question as Q WHERE NOT EXISTS ( SELECT * FROM SurveyStructure as S \
                    WHERE S.SurveyId = ' + str(surveyId) + ' AND S.QuestionId = Q.QuestionId)) as t UNION '
#       See HERE! ----------------------------------------------------------------------------------^
        cursor = sql_conn.cursor()
        cursor.execute(query_QuestionInSurvey)
...

Answer 2

我用另一種方式來執行我的 SQL 查詢： query_questionInSurvey = pd.read_sql_query(InSurveyList, sql_conn) 而不是 cursor.execute()