SQL Server varchar与nvarchar的意外“排序依据”差异

Question

I have encountered an unexpected difference between the results of 'order by' with varchar vs. nvarchar data. In both cases the data in question is from the old ASCII character set; the difference occurs in ordering data beginning with nnn vs -nnn, where n is a digit.

Below is SQL Server script which reproduces the problem; my test server is SQL 2016, but I have reproduced the problem in 2008 and 2012 as well. I have tried different collations with no effect (except Latin1_General_bin, see below). The script creates 2 sample tables similar to one in our application, one using varchar and the other nvarchar, and adds 7 rows of data.

CREATE TABLE [dbo].[_ValidationLists](
    [_FldNum] [int] NOT NULL,
    [_ValidationEntry] [varchar](250) NOT NULL,
 CONSTRAINT [PK__ValidationLists] PRIMARY KEY CLUSTERED 
(
    [_ValidationEntry] ASC,
    [_FldNum] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]
) ON [PRIMARY]

GO

CREATE TABLE [dbo].[_ValidationListsN](
    [_FldNum] [int] NOT NULL,
    [_ValidationEntry] [nvarchar](250) NOT NULL,
 CONSTRAINT [PK__ValidationListsN] PRIMARY KEY CLUSTERED 
(
    [_ValidationEntry] ASC,
    [_FldNum] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]
) ON [PRIMARY]

GO

INSERT INTO [_ValidationLists] (_fldnum, _ValidationEntry) VALUES (1,'-1'), (1,'-10'), (1,'-100'), (1,'0'), (1,'1'), (1,'10'), (1,'100')
INSERT INTO [_ValidationListsN] (_fldnum, _ValidationEntry) VALUES (1,N'-1'), (1,N'-10'), (1,N'-100'), (1,N'0'), (1,N'1'), (1,N'10'), (1,N'100')


select * from [_ValidationLists]
order by [_ValidationEntry] asc
select * from [_ValidationListsN]
order by [_ValidationEntry] asc

The results of the select statements are below. The first result, for the varchar, is what I expect (lexicographic sort); the second results I cannot explain. The first is what our customer base also expects, and we were caught be surprise by this result. (The customer data is unusual - typically this table is used for alpha data; and alpha data orders identically for both varchar and nvarchar).

Results are identical using N'...' to initialize _ValidationListsN rows. Original data had longer entries, such as '-100:Pass'; I have edited the data down to the least which demonstrate the problem.

Right-padding with blanks so all entries are same length has no effect.

Using COLLATE Latin1_General_bin reproduces lexicographic sorting, but is not acceptable because (for only one reason) we generally use case-insensitive sorts.

Our customer which reported this problem has only ASCII data, so we can fix them by recreating this table using varchar. I would love to know why nvarchar behaves this way, since the results seem incorrect to me, and if there is a way to get the ordering behavior we expect (the first case). At the least I have no idea why all entries which begin with '-' (ASCII 0X2d, dash or minus sign) do not order together.

_FldNum _ValidationEntry

1           -1  
1           -10  
1           -100  
1           0  
1           1  
1           10  
1           100  

(7 row(s) affected)

_FldNum _ValidationEntry

1           0  
1           1  
1           -1  
1           10  
1           -10  
1           100  
1           -100  

(7 row(s) affected)

Answer 1

When SQL Server resolves ORDER BY , it decide the order of rows not only from collation, but also, as you already discovered, from data type.

Both varchar and nchar are at the end only binaries. The problem is, that in varchar, "-" sign comes after numbers/chars (when it comes to binary notation), for nvarchar, it is the opposite. Check the internet for ASCI/UNICODE tables.

Because of that, since ORDER BY is in the end, comparing binary, your negative values comes first for nvarchar.

If you are more interested in how the data are actually stored, book "Microsoft SQL Server Internals" might be interesting for you. There is a whole section discussing this problem.

EDIT:

To see, how data are actually stored inside db, see this snippet:

SELECT 
   [_ValidationEntry], 
   CONVERT(binary(6), [_ValidationEntry]) AS [BinaryRepresentation] 
FROM [_ValidationLists]
--ORDER BY [_ValidationEntry] COLLATE Latin1_General_bin

SELECT 
   [_ValidationEntry], 
   CONVERT(binary(6), [_ValidationEntry]) AS [BinaryRepresentation] 
FROM [_ValidationListsN]
--ORDER BY [_ValidationEntry] COLLATE Latin1_General_bin

Result of the query is like that:

0      0x300000000000
1      0x310000000000
-1     0x2D3100000000
10     0x313000000000
-10    0x2D3130000000
100    0x313030000000
-100   0x2D3130300000

When using BIN collation, column is ordered by its binary representation, therefore, 2D minus sign comes first.