Elasticsearch本机脚本-评估索引文档的字段值

Question

I'm trying to modify the Cosine Similarity Script from imotov on Github . In his script, his docWeightSum only takes the term frequency (tf) of terms that are in the query, not all the terms in the document itself.

Take this example below. The docWeightSum would be 9 (4 for "I", 4 for "am", 1 for "Sam"). What I want to the docWeightSum to be is 10 (add 1 for "ham") because I want to normalize the dot product by both the magnitudes of two vectors.

doc: "I am am I ham Sam"

query: "Sam I am"

So I actually have 2 questions, as I index document into Elasticsearch like this:

POST /termscore/doc
{
   "text": "I am am I ham",
   "docWeightSum": 9
}

• Is there existing API to get the sum square of all tf for each indexed document, or to get tf of terms in the document that are not in the query? If not, then how can I compute this sum square?
• If I precompute the sum square of tf of each document and put into Elasticsearch along with the document content, as in the example above, then when computing the score, how can I access that "docWeightSum" value?

I am using Elasticsearch 1.7

Thanks,

Answer 1

To answer your question, it's possible, but it would be very inefficient to calculate docWeightSum in runtime. So, assuming that you precompute the value and index it in a separate field, you can access these values from a native script using doc lookup mechanism. If your calculations are not very complex you might be able to get by using field value factor in a function_score query and avoid writing your own script altogether.

Saying that, I suspect you are asking a wrong question. Instead of trying to implement it as a scoring script, I would suggest to look into creating your own custom SimilarityProvider. You will most likely find that most of the constructs that you are trying to shoehorn into score script are already there and much easier to implement and use.