Lucene.net result as string / parsing to slow

Question

I need some help to speed up the lucene.net search. We need the result in a string with ; as a seperator. The parsing of the topdocs takes to long:

    Dim resultDocs As TopDocs = indexSearch.Search(query, indexReader.MaxDoc())
    Dim hits As Object = resultDocs.ScoreDocs

    Dim strGetDocIDList As String = ""

    For Each sDoc As ScoreDoc In hits
        Dim documentFromSearcher As Document = indexSearch.Doc(sDoc.Doc)
        Dim contentValue As String = documentFromSearcher.Get("id")

        strGetDocIDList = strGetDocIDList + Path.GetFileName(contentValue) + ";"

    Next

    Return strGetDocIDList

How can we speed this up?

Regards Ingo

Answer 1

There are a few ways to tune performances for loading STORED fields in Lucene.

First, by default its loads every stored fields of the Document when you load it. Only store what you need and do not systematically store everything.

If you dont need to load all the stored fields for this particular query, try writing a FieldSelector to gain further control on field loading.

Finally, add the Field you load the stored data for more often before other stored fields in your Documents. Fields are loaded sequentially and in some cases, adding them first to Documents can speed up things a little bit.

FieldSelector API link

An article that may help you with implementing a FieldSelector