LINQ query with FirstOrDefault vs ToArray

Question

Using MySQL 5.6 and the following table structure:

CREATE TABLE `dataitem` (
  `AI` int(11) unsigned NOT NULL AUTO_INCREMENT,
  `ID` binary(16) NOT NULL,
  `OwnerID` binary(16) NOT NULL,
  `DataItemTimeUtc` datetime NOT NULL,
  `DataItemTimeLocal` datetime NOT NULL,
  `DataItemTimeMicroSeconds` int(11) NOT NULL,
  `DataItemArrivalTimeUtc` datetime NOT NULL DEFAULT '0001-01-01 00:00:00',
  `DataItemTimeTimeZoneID` binary(16) NOT NULL,
  `QuestionID` binary(16) NOT NULL,
  `QuestionHistoryID` binary(16) DEFAULT NULL,
  `QuestionAbsolutePositionID` varchar(1000) COLLATE utf8_unicode_ci DEFAULT NULL,
  `GroupSessionIDString` varchar(250) COLLATE utf8_unicode_ci DEFAULT NULL,
  `DataItemType` int(11) NOT NULL,
  `DataEntryDevice` varchar(250) COLLATE utf8_unicode_ci DEFAULT NULL,
  `DataEntryDeviceCradle` varchar(250) COLLATE utf8_unicode_ci DEFAULT NULL,
  `DataItemXml` longtext COLLATE utf8_unicode_ci NOT NULL,
  PRIMARY KEY (`AI`),
  UNIQUE KEY `dataitem_ID_UQ_Idx` (`ID`),
  KEY `dataitem_OwnerID_Idx` (`OwnerID`),
  KEY `dataitem_DataItemTimeUtc_Idx` (`DataItemTimeUtc`),
  KEY `dataitem_QuestionID_Idx` (`QuestionID`),
  KEY `dataitem_QuestionHistoryID_Idx` (`QuestionHistoryID`),
  KEY `dataitem_QuestionAbsolutePositionID_Idx` (`QuestionAbsolutePositionID`(255)),
  KEY `dataitem_DataItemType_Idx` (`DataItemType`)
) ENGINE=InnoDB AUTO_INCREMENT=23467 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;

I am experiencing something that I am struggling to understand. The following query causes a fatal error because it is taking forever to execute:

        Guid patientid = new Guid("cfed2acf-acbd-4ab2-8c23-7ab0b3a8cfa3");
        var latestRecord = (from f in QueryHelper.GetEntityTable<DataItem>()
                              where
                              f.OwnerID == patientid
                              && f.QuestionAbsolutePositionID == "5867FF5EC08B9C0422EFD1359B2802B29A8E167952D381EC70AE53CE6D4C9318"
                              orderby f.DataItemTimeUtc descending
                              select f.ID).FirstOrDefault();

However if I change .FirstOrDefault() to .ToArray() the query runs like a flash and returs 2 results. Can someone explain this?

SQL Query generated from .ToArray():

SELECT t0.`ID`
FROM `DataItem` AS t0
WHERE ((t0.`OwnerID` = @p0) AND (t0.`QuestionAbsolutePositionID` = @p1))
ORDER BY t0.`DataItemTimeUtc` DESC
-- p0 = [cfed2acf-acbd-4ab2-8c23-7ab0b3a8cfa3]
-- p1 = [5867FF5EC08B9C0422EFD1359B2802B29A8E167952D381EC70AE53CE6D4C9318]

SQL query generated from .FirstOrDefault():

SELECT t0.`ID`
FROM `DataItem` AS t0
WHERE ((t0.`OwnerID` = @p0) AND (t0.`QuestionAbsolutePositionID` = @p1))
ORDER BY t0.`DataItemTimeUtc` DESC
LIMIT 0, 1
-- p0 = [cfed2acf-acbd-4ab2-8c23-7ab0b3a8cfa3]
-- p1 = [5867FF5EC08B9C0422EFD1359B2802B29A8E167952D381EC70AE53CE6D4C9318]

Answer 1

First, figure out why QuestionAbsolutePositionID needs to be 1000 characters long. It it can be less than 256, make it so. If no, then ask yourself whether it can be changed to CHARACTER SET ascii . It looks like hex, which works fine with ascii . (Rarely do "ids" include accented letters, Cyrillic, Japanese, etc.) If neither of those 'fixes' are possible, can you upgrade to MySQL 5.7?

Once you have fixed the problem of index size (above), add this 'composite' (and 'covering') index; it should speed up the query:

INDEX(OwnerID, QuestionAbsolutePositionID, DataItemTimeUtc, ID)

(The first two columns can be in either order.)

If it does not help, then we need to discuss the @variables.