Implement "random" sort in hibernate search elasticsearch

Question

I want to achieve some kind of "random" sort using hibernate search elasticsearch library. What I'm doing is the following:

Implement a FieldComparator:

public class RandomOrderFieldComparator extends FieldComparator<Integer>  {

  private final Random randomGenerator = new Random();

  @Override
  public int compare(int slot1, int slot2) {
    return randomGenerator.nextInt();
  }

  @Override
  public void setTopValue(Integer value) {
    //not needed as the purpose is to generate random integers w
  }

  @Override
  public Integer value(int slot) {
    return randomGenerator.nextInt();
  }

  @Override
  public LeafFieldComparator getLeafComparator(LeafReaderContext context) throws IOException {
    return null;
  }
}

Implement FieldComparatorSource

public class SampleFieldComparatorSource extends FieldComparatorSource {

  @Override
  public FieldComparator<?> newComparator(String fieldname, int numHits, int sortPos, boolean reversed) throws IOException {
    return new RandomOrderFieldComparator();
  }
}

Finally create a custom SortField providing the FieldComparatorSource:

queryBuilder
        .sort()
        .byNative(
            new SortField(
                "id",
                new SampleFieldComparatorSource()
            )
        );

The problem is that it still generates a query using just a normal sort on the 'id' field and the comparator is never hit:

"sort": [
    {
      "id": {
        "order": "asc"
      }
    }
  ]

What am I doing wrong and what is the best way to implement a "random" sort using the hibernate search library?

Answer 1

Actually I found a way to do it with hibernate search passing directly a JSON and using painless script:

queryBuilder.sort()
          .byNative("_script", "{"
                                  + "\"type\" : \"number\","
                                  + "\"script\" : {"
                                  + "   \"lang\": \"painless\","
                                  + "   \"source\": \"new Random().nextInt()\""
                                  + "},"
                                  + "\"order\" : \"asc\""
                                + "}");

Answer 2

The Elasticsearch integration works by translating the Lucene objects into JSON and sending that to the Elasticsearch cluster. It works for simple things, such as sorts by field values, term queries, and so on, but it definitely cannot translate a Java object you implemented yourself.

Simply put, as soon as you use a custom implementation of a Lucene interface, you can be certain it won't work with the Elasticsearch integration. So your RandomOrderFieldComparator just won't work.

If you need to do advanced stuff that Hibernate Search doesn't expose through its APIs, such as this random sort, you will have to write the JSON sent to Elasticsearch by yourself. Stackoverflow provides various solutions to that problem.

EDIT: the rest of my answer was wrong. I had forgotten about the native sort feature, shame on me. See the other answer

Answer 3

I was unable to get seeding for pagination working using Hristo Angelov's answer. I had tried using new Random(long).nextInt() but it just kept generating the same order regardless of the seed value. Following the link in yrodiere's answer led me to this solution, which works flawlessly.

String fieldName = "id"; //replace with your search field
String seed = "hello world"; //generate a random string that persists across pages of the same search
sort = qb.sort()
         .byNative("_script", 
                   "{"
                 +    "\"script\" : \"(doc['" + fieldName + "'].value + '" + seed + "').hashCode()\","
                 +    "\"type\" : \"number\"," 
                 +    "\"order\" : \"asc\"" 
                 + "}")
         .createSort();