改善自动完成LINQ查询的性能

Question

I have some massive searches happening for my AutoComplete and was wondering if someone could give any ideas to improve the performance.

What happens:

1) At application launch I am saving all database entries on the memory.

2) User types in the search box to initiate AutoComplete:

$("#MatterCode").width(110).kendoAutoComplete({
        minLength: 3,
        delay: 10,
        dataTextField: "MatterCode",
        template: '<div class="autoCompleteResultsCode"> ${ data.ClientCode } - ${ data.MatterCode } - ${ data.ClientName } - ${ data.MatterName }</div>',
        dataSource: {
            serverFiltering: true,
            transport: {
                read: "/api/matter/AutoCompleteByCode",
                parameterMap: function() {
                    var matterCode = $("#MatterCode").val();
                    return { searchText: matterCode };
                }
            }
        }, //More Stuff here

3) It goes to my controller class:

public JsonResult AutoCompleteByCode(string searchText)
{
    if (string.IsNullOrEmpty(searchText))
    {
        Response.StatusCode = 500;
        return Json(new
        {
            Error = "search string can't be empty"
        });
    }

    var results = _publishedData.GetMattersForAutoCompleteByCode(searchText).Select(
            matter => new
            {
                MatterCode = matter.Code,
                MatterName = matter.Name,
                ClientCode = matter.Client.Code,
                ClientName = matter.Client.Name
            });
    return Json(results);
}

4) Which goes into the DAL (objects starting with '_' are Memory Objects)

public virtual IEnumerable<Matter> GetMattersForAutoCompleteByCode(string input)
{
    InvalidateCache();
    IEnumerable<Matter> results;
    //Searching Matter Object on all 4 given parameters by input.

    if (_lastMatters != null && input.StartsWith(_lastSearch) && _lastMatters.Any())
    {
        results = _lastMatters.Where(m => m.IsInputLike(input)).OrderBy(m => m.Code);
        _lastMatters = results;
    }
    else
    {
        results = _matters.Where(m => m.IsInputLike(input)).OrderBy(m => m.Code);
        _lastMatters = results;
    }

    _lastSearch = input;

    return results.Take(10).ToList();
}

5) isInputLike is an internal bool method

internal bool IsInputLike(string input)
{
    //Check to see if the input statement exists in any of the 4 fields
    bool check = (Code.ToLower().Contains(input.Trim().ToLower()) 
            || Name.ToLower().Contains(input.Trim().ToLower()) 
            || ClientCode.ToLower().Contains(input.Trim().ToLower()) 
            || ClientName.ToLower().Contains(input.Trim().ToLower()));

    return check;
}

Now the result set that I have to work with can range over 100,000. Now the first Autocomplete of any new query has to search through 400,000 records and I can't think of a way to improve the performance without sacrificing the feature.

Any ideas? Is SQL stored proc calls faster than LINQ?

Answer 1

I'm not much of an asp/http guy but when I see this:

internal bool IsInputLike(string input)
{
    //Check to see if the input statement exists in any of the 4 fields
    bool check = (Code.ToLower().Contains(input.Trim().ToLower()) 
        || Name.ToLower().Contains(input.Trim().ToLower()) 
        || ClientCode.ToLower().Contains(input.Trim().ToLower()) 
        || ClientName.ToLower().Contains(input.Trim().ToLower()));

    return check;
}

I think you are creating a lot of new string; and that has to take some time. Try this and see if this improves your performance

var inp = input.Trim();
bool chk = (Code.IndexOf(inp, StringComparison.CurrentCultureIgnoreCase) > -1)
                || (Name.IndexOf(inp, StringComparison.CurrentCultureIgnoreCase) > -1)
                || (ClientCode.IndexOf(inp, StringComparison.CurrentCultureIgnoreCase) > -1)
                || (ClientName.IndexOf(inp, StringComparison.CurrentCultureIgnoreCase) > -1);

This first line (that creates inp) isn't that important since the compiler should optimize repeated usage, but I think it reads better.

The IndexOf method will not create new strings and with the StringComparison parameter you can avoid creating all the ToLower strings.

Answer 2

I think the main issue here is you placing the 400k objects in memory to start with. SQL is not all that slow, it's better to start with a limited set of data in the first place.

one obvious optimisation is:

internal bool IsInputLike(string input)
{
    string input = input.Trim().ToLower();
    //Check to see if the input statement exists in any of the 4 fields
    bool check = (Code.ToLower().Contains(input) 
            || Name.ToLower().Contains(input) 
            || ClientCode.ToLower().Contains(input) 
            || ClientName.ToLower().Contains(input));

    return check;
}

but personally, I would keep the data where it belongs, in the SQL server (if that's what you are using). Some indexing and the proper queries could make this faster.

When I see this code I start wondering:

public virtual IEnumerable<Matter> GetMattersForAutoCompleteByCode(string input)
{
    InvalidateCache();
    IEnumerable<Matter> results;
    //Searching Matter Object on all 4 given parameters by input.

    if (_lastMatters != null && input.StartsWith(_lastSearch) && _lastMatters.Any())
    {
        results = _lastMatters.Where(m => m.IsInputLike(input)).OrderBy(m => m.Code);
        _lastMatters = results;
    }
    else
    {
        results = _matters.Where(m => m.IsInputLike(input)).OrderBy(m => m.Code);
        _lastMatters = results;
    }

    _lastSearch = input;

    return results.Take(10).ToList();
}

why do you need to order? Why does a dropdown autocomplete need to filter on 4 items? if you only take 10 anyway can't you just not order? See if removing the orderby gives you any better results, especially in the else statement where you'll have many results.

personally i'd go all in for LINQ to SQL and let the SQL server do the searching. optimize the indexing on this table and it'll be much faster.

Answer 3

Well i recommend you to create a view that contains all of the names eg (code, name, Clientcode, ClientName) into a single column concatenated say FullName and replace your IsInputLike(..) as below:

internal bool IsInputLike(string input)
{
    //Check to see if the input statement exists in any of the 4 fields
    return FullName.Contains(input);

}