Search multiple models (Elasticsearch)

Question

i added a second model to my app and now i also want to search trough the fields for that model and return it in the same view as my first model.

views.py

#Elasticsearch
def globalsearch_elastic(request):
    qs = request.GET.get('qs')
    page = request.GET.get('page')
    if qs:
        qs = PostDocument.search().query("multi_match", query=qs, fields=["title", "content", "tag"])
        qs = qs.to_queryset()
    else:
        qs = ''
    paginator = Paginator(qs, 10)  # Show 10 results per page
    try:
        qs = paginator.page(page)
    except PageNotAnInteger:
        qs = paginator.page(1)
    except EmptyPage:
        qs = paginator.page(paginator.num_pages)
    return render(request, 'MyProject/search/search_results_elastic.html', {'object_list':qs})

currently only PostDocument get searched for a match, how do i add a second one here from documents.py in this function? or to be more specific, what do i have to change in this line to Search multiple documents?

qs = PostDocument.search().query("multi_match", query=qs, fields=["title", "content", "tag"])

this is where "qs" comes from (base.html):

<div class="globalsearch">
        <form id="searchform" action="{% url 'search' %}" method="get" accept-charset="utf-8">
            <div class="form-row align-items-center">
                <input class="class-search-input-fields" id="searchbox" name="qs" required="required" type="text" placeholder="Search ..."><a>in</a>
                <div class="custom-dropdown">
                    <a>{{ categorysearch_form.category }}</a>
                </div>
                <button class="btn btn-dark" type="submit">
                    <i class="fa fa-search"></i>
                </button>
            </div>
        </form>
    </div>

documents.py:

Thanks in advance

Answer 1

In a match query you already search all the tokens in the query as an OR query in SQL . In your case, if you have a multimatch query against 3 field means that you are searching for a match of any of your token in your query against any of that 3 fields - if you haven't assigned any specific mapping to ES, it means that your text fields have processed with a standard analyzer please read here that breaks your string in tokens on whitespace. So to add a new key to be queryied, just concatenate a new value to the string qs :

from document import DocClass

def globalsearch_elastic(request):
    qs = request.GET.get('qs')
    document = DocClass()
    document_value = document.my_function() # you invoke a function in a class inside document.py that returns to you a string
    page = request.GET.get('page')
    if any([qs, document_value]):
        queryable = " ".join([qs, document_value])
        qs = PostDocument.search().query("multi_match", query=queryable, fields=["title", "content", "tag"])
        qs = qs.to_queryset()
    else:
        return "No values to query"

Answer 2

Got it working like this:

def globalsearch_elastic(request):
    qs = request.GET.get('qs')
    page = request.GET.get('page')
    if qs:
        post = PostDocument.search().query("multi_match", query=qs, fields=["title", "content", "tag"]).to_queryset()
        post_extra = PostExtraDocument.search().query("multi_match", query=qs, fields=["title", "content_preview", "tag"]).to_queryset()
        qs = list(
            sorted(
                chain(post, post_extra),
                key=lambda objects: objects.pk
            ))
    else:
        qs = ''
    paginator = Paginator(qs, 10)  # Show 10 results per page
    try:
        qs = paginator.page(page)
    except PageNotAnInteger:
        qs = paginator.page(1)
    except EmptyPage:
        qs = paginator.page(paginator.num_pages)
    return render(request, 'MyProject/search/search_results_elastic.html', {'object_list': qs})

i know that this is maybe not "best-practice" because i always have to load all posts, sort them afterwards and display them by creation-date (pk).

Anyways, from my point of view it's fine if you don't have to search millions f posts ... Beside that you still have the power of elastic even if that does not sort my qs list of posts

Answer 3

The accepted answer is correct but not efficient because it fetches rows from the database to convert the results to a querysets and finally chain and sort...

I hope my answer will help others. I'm using django-oscar and dajngo-elasticsearch-dsl and I assume that you're at least using elasticsearch-dsl. So you can simply do

# *********************************************** documents.py in user module
from django_elasticsearch_dsl import Document
from django_elasticsearch_dsl.registries import registry
from oscar.core.compat import get_user_model

User = get_user_model()

@registry.register_document
class UserDocument(Document):

    class Index:
        name = 'users'
        settings = {
            'number_of_shards': 1,
            'number_of_replicas': 0
        }

    class Django:
        model = User # <-- User model class 

        fields = [
            'first_name',
            'last_name',
            'username'
        ]


# *********************************************** documents.py in product module
from django_elasticsearch_dsl import Document, TextField
from django_elasticsearch_dsl.registries import registry
from oscar.core.loading import get_model

Product = get_model('catalogue', 'product')

@registry.register_document
class ProductDocument(Document):

    upc = TextField(attr='upc', required=False)

    class Index:
        name = 'products'
        settings = {
            'number_of_shards': 1,
            'number_of_replicas': 0
        }

    class Django:
        model = Product # <-- Product model class

        fields = [
            'title',
            'description'
        ]


# *********************************************** search method
from django.shortcuts import render
from elasticsearch_dsl import Search

def global_search(request):

    q = request.GET.get('q')
    objects = ''

    if q:
        search = Search(index=['users', 'products'])
        objects = search.query("multi_match", query=q, fields=['first_name', 'last_name', 'username', 'title', 'description', 'upc'])

    return render(request, 'oscar/search/search.html', {'objects': objects})