How to get all indices with Elastic's High Level Rest Client?

Question

I want a nice, quick and easy way to get all of the indices in elasticsearch using the their Java REST client . I am currently able to do this by grabbing their lower level client, like this:

public void fetchIndices() throws IOException {
    List<String> indices = null;

    RestClient restClient = client.getLowLevelClient();
    Response response = null;
    try {
        response = restClient.performRequest("GET", "/_cat/indices?v");
    } catch (IOException e) {
        LOGGER.log(Level.WARNING, e.toString(), e);
    }

    InputStream inputStream = null;
    if (response != null) {
        try {
            inputStream = response.getEntity().getContent();
        } catch (IOException e) {
            LOGGER.log(Level.WARNING, e.toString(), e);
        }
    }

    if (inputStream != null) {
        InputStreamReader inputStreamReader = new InputStreamReader(inputStream);
        BufferedReader bufferedReader = new BufferedReader(inputStreamReader);

        indices = new ArrayList<>();
        String line;
        while ((line = bufferedReader.readLine()) != null) {
            // Get tokens with no whitespace
            String[] tokens = line.split("\\s+");
            for (String token : tokens) {
                // TODO - make the startsWith() token configurable
                if (token.startsWith(SOME_TOKEN)) {
                    LOGGER.log(Level.INFO, "Found elasticsearch index " + token);
                    indices.add(token);
                    break;
                }
            }
        }
    }

    // Only update if we got data back from our REST call
    if (indices != null) {
        this.indices = indices;
    }
}

Essentially I just call the /_cat/indices?v endpoint as recommended in their docs . This works fine, but I was wondering if there was a nicer way to do this using the Java API. I can't seem to find a way in their current API, but wondering if anyone knows something I don't. Having to work with InputStream s and the various Reader s isn't necessarily terrible, but just want to clean up the hacky string parsing.

Answer 1

As of Elasticsearch 7.5.0 you can use the following to retrieve all indices:

    GetIndexRequest request = new GetIndexRequest("*");
    GetIndexResponse response = client.indices().get(request, RequestOptions.DEFAULT);
    String[] indices = response.getIndices();

Answer 2

At the moment, the high-level REST client doesn't support this. You can keep calling the _cat/indices API using the low-level client but try adding &format=json in the query string parameters. That way you'll get the same information but formatted as JSON which is much easier to parse (eg with the Jackson library):

List<String> indices = null;

RestClient restClient = client.getLowLevelClient();
Response response = null;
try {
    response = restClient.performRequest("GET", "/_cat/indices?v&format=json");
} catch (IOException e) {
    LOGGER.log(Level.WARNING, e.toString(), e);
}

// parse the JSON response
List<Map<String, String>> list = null;
if (response != null) {
    String rawBody = EntityUtils.toString(response.getEntity());
    TypeReference<List<HashMap<String, String>>> typeRef = new TypeReference<List<HashMap<String, String>>>() {};
    list = mapper.readValue(rawBody, typeRef);
}

// get the index names
if (list != null) {
    indices = list.stream()
        .map(x -> x.get("index"))
        .collect(Collectors.toList());
}

// Only update if we got data back from our REST call
if (indices != null) {
    this.indices = indices;
}

Note: Here is the roadmap for the High-Level REST client: https://github.com/elastic/elasticsearch/issues/27205

Answer 3

on es 6.8, use * or _all in GetIndexRequest when there is no index throws NoSuchIndexException.

I found this is the safer, no throw way:

    GetMappingsResponse response = esClient.indices().getMapping(new GetMappingsRequest().indices("*"),
            RequestOptions.DEFAULT);
    return new ArrayList<>(response.mappings().keySet());

Answer 4

Try using: /_cat/indices?h=i

InputStream inputStream = restHighLevelClient.getLowLevelClient()
.performRequest("GET", "/_cat/indices?h=i")
.getHttpResponse()
.getEntity()
.getContent();

List<String> indexes = new BufferedReader(new InputStreamReader(inputStream))
    .lines()
    .collect(Collectors.toList());

Also, if you want to search using a regex: /_cat/indices?h=i&index=test*

Answer 5

For ES version 7.3
Unfortunately, the answer of the developers in the Elasticsearch road map page is: "Don't think it makes sense to add support for cat API" So I see only one solution - using Low-level client.

As an example in my application I use:

/**
     Get Low level client from High level client. You can defined it directly as Low level:
     RestClient lowLevelClient = RestClient.builder(
     new HttpHost("localhost", 9200, "http"),
     new HttpHost("localhost", 9201, "http")).build();
     */
    RestClient lowLevelClient = client.getLowLevelClient();
    List<String> indexList = new LinkedList<>();

    // Request from _cat/indices. You can change 'h' flags for receiving additional index parameters
    Response response = lowLevelClient.performRequest(new Request("GET", "/_cat/indices?h=index&format=json"));

    // parse the JSON response
    List<Map<String, String>> listOfIndicesFromEs = null;
    if (response != null) {
        String rawBody = EntityUtils.toString(response.getEntity());
        TypeReference<List<HashMap<String, String>>> typeRef = new TypeReference<List<HashMap<String, String>>>() {
        };
        listOfIndicesFromEs = mapper.readValue(rawBody, typeRef);
    }

    // get the index names
    if (listOfIndicesFromEs != null) {
        indexList = listOfIndicesFromEs.stream()
                .map(index -> index.get("index"))
                .collect(Collectors.toList());
    }
    return indexList;

mapper - from Jackson library

ObjectMapper mapper = new ObjectMapper();

Hope this code will help you:)