SQL standard select current records from an audit log question

Question

My memory is failing me. I have a simple audit log table based on a trigger:

ID int (identity, PK)
CustomerID int
Name varchar(255)
Address varchar(255)
AuditDateTime datetime
AuditCode char(1)


It has data like this:

ID CustomerID Name Address AuditDateTime AuditCode 1 123 Bob 123 Internet Way 2009-07-17 13:18:06.353 I 2 123 Bob 123 Internet Way 2009-07-17 13:19:02.117 D 3 123 Jerry 123 Internet Way 2009-07-17 13:36:03.517 I 4 123 Bob 123 My Edited Way 2009-07-17 13:36:08.050 U 5 100 Arnold 100 SkyNet Way 2009-07-17 13:36:18.607 I 6 100 Nicky 100 Star Way 2009-07-17 13:36:25.920 U 7 110 Blondie 110 Another Way 2009-07-17 13:36:42.313 I 8 113 Sally 113 Yet another Way 2009-07-17 13:36:57.627 I


What would be the efficient select statement be to get all most current records between a start and end time? FYI: I for insert, D for delete, and U for update.

Am I missing anything in the audit table? My next step is to create an audit table that only records changes, yet you can extract the most recent records for the given time frame. For the life of me I cannot find it on any search engine easily. Links would work too. Thanks for the help.

Answer 1

Another (better?) method to keep audit history is to use a 'startDate' and 'endDate' column rather than an auditDateTime and AuditCode column. This is often the approach in tracking Type 2 changes (new versions of a row) in data warehouses.

This lets you more directly select the current rows (WHERE endDate is NULL), and you will not need to treat updates differently than inserts or deletes. You simply have three cases:

• Insert: copy the full row along with a start date and NULL end date
• Delete: set the End Date of the existing current row (endDate is NULL)
• Update: do a Delete then Insert

Your select would simply be:

select * from AuditTable where endDate is NULL

Anyway, here's my query for your existing schema:

declare @from datetime
declare @to datetime

select b.* from (
  select
    customerId
    max(auditdatetime) 'auditDateTime'
  from
    AuditTable
  where
    auditcode in ('I', 'U')
    and auditdatetime between @from and @to
  group by customerId
  having 
    /* rely on "current" being defined as INSERTS > DELETES */
    sum(case when auditcode = 'I' then 1 else 0 end) > 
    sum(case when auditcode = 'D' then 1 else 0 end)
) a
cross apply(
  select top 1 customerId, name, address, auditdateTime
  from AuditTable
  where auditdatetime = a.auditdatetime and customerId = a.customerId
) b

References

A cribsheet for data warehouses , but has a good section on type 2 changes (what you want to track)

MSDN page on data warehousing

Answer 2

Ok, a couple of things for audit log tables.

For most applications, we want audit tables to be extremely quick on insertion.

If the audit log is truly for diagnostic or for very irregular audit reasons, then the quickest insertion criteria is to make the table physically ordered upon insertion time.

And this means to put the audit time as the first column of the clustered index, eg

create unique clustered index idx_mytable on mytable(AuditDateTime, ID)

This will allow for extremely efficient select queries upon AuditDateTime O(log n), and O(1) insertions.

If you wish to look up your audit table on a per CustomerID basis, then you will need to compromise.

You may add a nonclustered index upon (CustomerID, AuditDateTime), which will allow for O(log n) lookup of per-customer audit history, however the cost will be the maintenance of that nonclustered index upon insertion - that maintenance will be O(log n) conversely.

However that insertion time penalty may be preferable to the table scan (that is, O(n) time complexity cost) that you will need to pay if you don't have an index on CustomerID and this is a regular query that is performed. An O(n) lookup which locks the table for the writing process for an irregular query may block up writers, so it is sometimes in writers' interests to be slightly slower if it guarantees that readers aren't going to be blocking their commits, because readers need to table scan because of a lack of a good index to support them....

---

Addition: if you are looking to restrict to a given timeframe, the most important thing first of all is the index upon AuditDateTime. And make it clustered as you are inserting in AuditDateTime order. This is the biggest thing you can do to make your query efficient from the start.

Next, if you are looking for the most recent update for all CustomerID's within a given timespan, well thereafter a full scan of the data, restricted by insertion date, is required.

You will need to do a subquery upon your audit table, between the range,

select CustomerID, max(AuditDateTime) MaxAuditDateTime 
from AuditTrail 
where AuditDateTime >= @begin and Audit DateTime <= @end

and then incorporate that into your select query proper, eg.

select AuditTrail.* from AuditTrail
inner join 
    (select CustomerID, max(AuditDateTime) MaxAuditDateTime 
     from AuditTrail 
     where AuditDateTime >= @begin and Audit DateTime <= @end
    ) filtration
    on filtration.CustomerID = AuditTrail.CustomerID and 
       filtration.AuditDateTime = AuditTrail.AuditDateTime

Answer 3

Another approach is using a sub select

select a.ID
       , a.CustomerID 
       , a.Name
       , a.Address
       , a.AuditDateTime
       , a.AuditCode
from   myauditlogtable a,
       (select s.id as maxid,max(s.AuditDateTime) 
                 from myauditlogtable as s 
                 group by maxid) 
        as subq
where subq.maxid=a.id;

Answer 4

start and end time? eg as in between 1am to 3am
or start and end date time? eg as in 2009-07-17 13:36 to 2009-07-18 13:36