简体   繁体   中英

Incorrect float subtraction result

DataSet

   id   qty CheckIn CheckOut
   5    10      1     0
   5    10      0     1
   5    1.6     1     0
   5    0.4     0     1
   5    0.4     0     1
   5    0.4     0     1

I am trying to select remaining quantity of id 5 .

Query:

Select 
    (select SUM(qty) from tbl where id = 5 and CheckIn = 1) -
    (select SUM(qty) from tbl where id = 5 and CheckOut = 1)

Expected Output: 4.0

Original Output: 3.9999999

Even the individual queries return perfect values:

select SUM(qty) from tbl where id = 5 and CheckIn = 1  -- returns 11.6
select SUM(qty) from tbl where id = 5 and CheckOut = 1 -- returns 11.2

But subtraction gives 3.9999999


You can generate data using the below query:

Here is the query to generate data set

CREATE TABLE [dbo].[tbl](
    [id] [int] NOT NULL,
    [qty] [float] NULL,
    [CheckIn] [bit] NULL,
    [CheckOut] [bit] NULL
) ON [PRIMARY]

GO
INSERT [dbo].[tbl] ([id], [qty], [CheckIn], [CheckOut]) VALUES (5, 10, 1, 0)
GO
INSERT [dbo].[tbl] ([id], [qty], [CheckIn], [CheckOut]) VALUES (5, 10, 0, 1)
GO
INSERT [dbo].[tbl] ([id], [qty], [CheckIn], [CheckOut]) VALUES (5, 1.6, 1, 0)
GO
INSERT [dbo].[tbl] ([id], [qty], [CheckIn], [CheckOut]) VALUES (5, 0.4, 0, 1)
GO
INSERT [dbo].[tbl] ([id], [qty], [CheckIn], [CheckOut]) VALUES (5, 0.4, 0, 1)
GO
INSERT [dbo].[tbl] ([id], [qty], [CheckIn], [CheckOut]) VALUES (5, 0.4, 0, 1)
GO
SELECT
    CAST(SUM(CASE WHEN CheckIn = 1 THEN qty END) AS MONEY)
    -
    CAST(SUM(CASE WHEN CheckOut = 1 THEN qty END) AS MONEY)
FROM tbl
WHERE id = 5

问题通过将数据类型更改为十进制来解决

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM