Efficient .includes() search in array of objects

Question

I'm writing a project which holds a big hardcoded array of objects as a database. Each object has some properties such as title and content. One of my tasks is trying to improve an existing search, which returns all the objects that include the keyword in the title or the content.

Right now I'm using the obvious solution of filtering, but the task is about handling millions of objects and they ask me how can I use a more efficient search to find the right objects. The code is written in Typescript/Javascript.

The objects look like this:

    {
        "id": "ab6fc754-0e01-5cfb-84b9-cf37c1c0cdb5",
        "title": "Collapse/expand code affecting the page footer\n",
        "content": "I have placed collapse/expand code onto one of my pages, but the footer now has a huge white space above it in the preview. Is there a way to fix this?",
        "userEmail": "susjorna@jep.sa",
        "labels": ["Collapse", "Expand"],
        "creationTime": 1517833093439
    },

Right now the simple search is this:

    paginatedData = paginatedData.filter((ticket: Ticket) => (ticket.content.toLowerCase() + ticket.title.toLowerCase()).includes(searchTerm.toLowerCase()))

But I probably need to rewrite and restructure the database(I can't change the DB itself, but probably do some kind of preprocessing).

One idea I had is trying to store the object and a database like Mongo, but I'm not sure if it gives me the option to search much faster the kind of search I need.

Answer 1

There are a few small improvements that can be made on the JS side:

• Call toLowerCase on the string being searched for once , instead of inside every callback:

const lowerSearch = searchTerm.toLowerCase();
const filteredData = paginatedData.filter(ticket => (ticket.content.toLowerCase() + ticket.title.toLowerCase()).includes(lowerSearch));

• Depending on the sort of strings you're working with, it might be faster to call .includes on the content and title separately, instead of concatenating the strings. Try it, you might see a slight performance difference:

const lowerSearch = searchTerm.toLowerCase();
const filteredData = paginatedData.filter(ticket => (
    ticket.content.toLowerCase().includes(lowerSearch) ||
    ticket.title.toLowerCase().includes(lowerSearch)
));

• An option that will increase the size of the database and memory usage in exchange for quicker operations would be to store the lower-cased versions of the strings, so you don't have to call toLowerCase on every string every time something gets searched for:

const lowerSearch = searchTerm.toLowerCase();
const filteredData = paginatedData.filter(ticket => (
    ticket.contentLower.includes(lowerSearch) ||
    ticket.titleLower.includes(lowerSearch)
));

But if you're dealing with huge numbers of objects, it would make more sense for the search functionality to be implemented in a database designed for this sort of thing, rather than filtering it in JavaScript.