Limiting values calculated and returned by Apache Solr Statscomponent

Question

Is there a way to configure solr statscomponent http://bit.ly/T4PLnl to NOT include the "expensive" sumOfSquares and stddev ?

We only need sum, min,max,mean,count and missing.

Answer 1

Use the AnalyticsComponent , which supports the following:

• All functionality of StatsComponent ( SOLR-4499 )
• Field Faceting ( SOLR-3435 )
• Support for limit
• Sorting (bucket name or any stat in the bucket
• Support for offset
• Range Faceting
• Supports all options of standard range faceting
• Query Faceting ( SOLR-2925 )
• Ability to use overall/field facet statistics as input to range/query faceting (ie calc min/max date and then facet over that range
• Support for more complex aggregate/mapping operations ( SOLR-1622 )
• Aggregations: min, max, sum, sum-of-square, count, missing, stddev, mean, median, percentiles
• Operations: negation, abs, add, multiply, divide, power, log, date math, string reversal, string concat

Usage:

Turning it on
Parameter Default Description
olap=[true/false] Enable enhanced analytics functionality

References

• SOLR-5302: Analytics Component

• Search Analytics Component Guide (pdf)

• Solr Source: AnalyticsParams.java

• Next-generation search and analytics with Apache Lucene and Solr 4

• RequestHandlers and SearchComponents in SolrConfig

• Search Analytics Component: Presented by Steven Bower, Bloomberg LP

Answer 2

This is something I tried once. All I cared about was sum, count, and missing, and still profiling showed that the stats are so cheap to compute as to be free. Consider that sumOfSquares requires O(1) space and O(n) time, the same as sum, min, max, mean, count, and missing. Whether you compute sumOfSquares or not, StatsComponent still has to iterate over the result set and fetch the field value for each doc. ParseDouble is overwhelmingly the hotspot for non-faceted stats, at least if you're using a string field!