防止重复SQL Server 2008和经典ASP

Question

I am importing records from Excel and I want to avoid duplicates. In ASP Classic I have written a function that checks the database for duplicates. If it finds one it adds a number on the end of the username and checks again if the username is unique, eg petejones becomes petejones1. Unfortunately this script is soooooo slow as the data base has around 150k records and it takes forever to search for uniqueness. Is there a way to do the same directly in SQL Server 2008 in T-SQL? so the whole process will be wicked quick. Is there a make unique process?

Here is the function in classic ASP.. I know there are better ways to do this, so dont laugh at my scripting.

FUNCTION CreateUniqueUsername(str)
  SET DbConn = Server.CreateObject("ADODB.connection")
  DbConn.Open DSN_LINK
  nCounter = 0
  Unique = ""
  IF InStr(str, "@") > 0 THEN
     strUsername = Left(str,InStr(str, "@")-1)
  ELSE
     strUsername = str
  END IF
  strUsername = FormatUsername(strUsername)
  strSQL = "SELECT UserName FROM Member WHERE UserName = '" & strUsername & "';"
  SET rs = DbConn.Execute(strSQL)
  IF rs.EOF AND rs.BOF THEN
    nFinalUsername = strUsername
  ELSE
    DO UNTIL Unique = true
      nCounter = nCounter + 1
      nFinalUsername = strUsername & nCounter
      strSQL2 = "SELECT UserName FROM Member WHERE UserName = '" & nFinalUsername & " ' "
      SET objRS = DbConn.Execute(strSQL2)
      IF objRS.EOF THEN
        Unique = true
      ELSE
        intCount = intCount
      END IF
    LOOP
    objRS.Close
    SET objRS = Nothing 
  END IF
  rs.Close
  SET rs = Nothing 
  SET DbConn = Nothing
  CreateUniqueUsername = nFinalUsername
END FUNCTION

FUNCTION FormatUsername(str)
  Dim OutStr
  IF ISNULL(str) THEN EXIT FUNCTION
  OutStr = lCase(Trim(str))
  OutStr = Replace(OutStr, "’", "")
  OutStr = Replace(OutStr, "”", "")
  OutStr = Replace(OutStr, "'","")
  OutStr = Replace(OutStr, "&","and")
  OutStr = Replace(OutStr, "'", "")
  OutStr = Replace(OutStr, "*", "")
  OutStr = Replace(OutStr, ".", "")
  OutStr = Replace(OutStr, ",", "")
  OutStr = Replace(OutStr, CHR(34),"")
  OutStr = Replace(OutStr, " ","")
  OutStr = Replace(OutStr, "|","")
  OutStr = Replace(OutStr, "&","")
  OutStr = Replace(OutStr, "[","")
  OutStr = Replace(OutStr, ";", "")
  OutStr = Replace(OutStr, "]","")
  OutStr = Replace(OutStr, "(","")
  OutStr = Replace(OutStr, ")","")
  OutStr = Replace(OutStr, "{","")
  OutStr = Replace(OutStr, "}","")
  OutStr = Replace(OutStr, ":","")
  OutStr = Replace(OutStr, "/","")
  OutStr = Replace(OutStr, "\","")
  OutStr = Replace(OutStr, "?","")
  OutStr = Replace(OutStr, "@","")
  OutStr = Replace(OutStr, "!","")
  OutStr = Replace(OutStr, "_","")
  OutStr = Replace(OutStr, "''","")
  OutStr = Replace(OutStr, "%","")
  OutStr = Replace(OutStr, "#","")
  FormatUsername = OutStr
END FUNCTION

Any assistance would be greatly appreciated as I am still learning SQL.

Answer 1

You could do this in SQL. This looks for a matching name. If a match is found, then it gets the maximum number currently appended to it and adds one. So at most it does two SELECTS . Should be faster when there are lots of duplicates.

-- example table
declare @Member table(ID int identity, UserName varchar(80))
insert @Member values('Pete')
insert @Member values('Jill')
insert @Member values('Bob')
insert @Member values('Sam')
insert @Member values('Pete1')
insert @Member values('Pete2')
insert @Member values('Pete3')
insert @Member values('Bob1')


declare @UserName varchar(80), @FinalUserName varchar(80)
set @UserName = 'Pete'

set @FinalUserName = @UserName
if(exists(SELECT 1 FROM @Member WHERE left(UserName,len(@UserName)) = @UserName))
begin
    SELECT 
        @FinalUserName = @UserName + convert(varchar(12),max(substring(UserName,len(@UserName)+1,99)+1)) 
    FROM @Member 
    WHERE left(UserName,len(@UserName)) = @UserName
end

SELECT @FinalUserName 

Answer 2

This cumbersome expression will retrieve first available user name. If there is a user having the same name and the rest of user's name is a number, expression will return username concatenated with next number. If such a username cannot be found, expression will return this username.

You might replace each '@username' with actual value or, better, use SqlCommand.ExecuteScalar . SqlCommand will allow the use of parameters, which is better solution because you don't have to concatenate ugly strings and because they prevent use of Sql Injection .

select @username 
     + isnull(convert (varchar (10),
         max (case when isnumeric (substring (m.Username, len (@username) + 1, 100)) = 1
                   then cast (substring (m.Username, len (@username) + 1, 100) as int) 
                   else (case when m.username = @username then 0 end)  
                   end) 
       + 1), '') UserName
from @members m
where m.username like @username + '%'

Here is a Sql Fiddle testing ground . Replace set @username = 'aa' with other usernames to see results.

Answer 3

This could be achived by inserting into a staging table that allows the duplicates, then transferring from the staging table into your main table, resolving the duplicates in the process.

INSERT INTO MainTable (Column1, Column2, UniqueName)
SELECT  Column1,
        Column2,
        UserName + ISNULL(CONVERT(VARCHAR, NULLIF(RowNumber, 0)), '') [UniqueName]
FROM    (   SELECT  *, *, ROW_NUMBER() OVER (PARTITION BY UserName ORDER BY Column1) - 1 [Rownumber]
            FROM    StagingTable
        ) staging

The important parts of this statement are:

ROW_NUMBER() OVER (PARTITION BY UserName ORDER BY Column1) - 1

This gives each row a row number (obviously). The PARTITION BY works like a group by, this basically means the row count will reset to 1 when username changes. The ORDER BY part determines which duplicate username should be row 1, which should be row 2 etc. ROW_NUMBER() starts at 1, so I have deducted 1 from it so it starts at 0.

Nextit is a matter of combining this row number with the username:

UserName + CONVERT(VARCHAR, RowNumber) [UniqueName]

This would yield Username0, Username1, username2... so the next step is to make "username0" only appear as username, to give a list of Username, username1, username2:

UserName + ISNULL(CONVERT(VARCHAR, NULLIF(RowNumber, 0)), '') [UniqueName]

This basically says if the rownumber is 0 then become null, then if the outcome of this is null then become ''.