如何将数据从 Java 发送到 Elastic Cloud？

Question

我想在 Java 应用程序中将一些数据插入（索引）到Elastic Cloud中运行的 Elastic Search 中。

为此，我编写了以下代码：

void sendStuffToElasticSearch() {
    RestHighLevelClient client = null;
    try {
        client = new RestHighLevelClient(
                RestClient.builder(CLOUD_ID)
        );

        RequestOptions.Builder builder = RequestOptions.DEFAULT.toBuilder();
        builder.addHeader("Authorization", String.format("ApiKey %s",
                API_KEY));
        final RequestOptions requestOptions = builder.build();

        IndexRequest request = new IndexRequest("posts");
        request.id("1");
        String jsonString = "{" +
                "\"user\":\"kimchy\"," +
                "\"postDate\":\"2013-01-30\"," +
                "\"message\":\"trying out Elasticsearch\"" +
                "}";
        request.source(jsonString, XContentType.JSON);
        IndexResponse indexResponse = client.index(request, requestOptions);
        System.out.println("indexResponse");
    } catch (IOException e) {
        e.printStackTrace();
    } finally {
        IOUtils.closeQuietly(client);
    }
}

API_KEY是我根据本教程生成的密钥，它还说我需要在Authorization header 中以以下格式发送它： Authorization: ApiKey $EC_API_KEY 。

当我运行上面的代码时，我收到以下错误：

org.elasticsearch.client.ResponseException: method [PUT], host [https://XXXXXXXXXX:9243], URI [/posts/_doc/1?timeout=1m], status line [HTTP/1.1 401 Unauthorized]
{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/posts/_doc/1?timeout=1m]","header":{"WWW-Authenticate":["Basic realm=\"security\" charset=\"UTF-8\"","Bearer realm=\"security\"","ApiKey"]}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/posts/_doc/1?timeout=1m]","header":{"WWW-Authenticate":["Basic realm=\"security\" charset=\"UTF-8\"","Bearer realm=\"security\"","ApiKey"]}},"status":401}
        at org.elasticsearch.client.RestClient.convertResponse(RestClient.java:326)
        at org.elasticsearch.client.RestClient.performRequest(RestClient.java:296)
        at org.elasticsearch.client.RestClient.performRequest(RestClient.java:270)
        at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1621)
        ... 30 more

如何解决这个问题，即以 Elastic Cloud 期望的方式提供所有与身份验证相关的数据？

我正在使用以下库：

    <properties>
        [...]
        <elastic-search-client.version>7.11.1</elastic-search-client.version>
    </properties>

        <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>elasticsearch-rest-client</artifactId>
            <version>${elastic-search-client.version}</version>
        </dependency>

        <dependency>
            <groupId>org.elasticsearch</groupId>
            <artifactId>elasticsearch</artifactId>
            <version>${elastic-search-client.version}</version>
        </dependency>
        <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>elasticsearch-rest-high-level-client</artifactId>
            <version>${elastic-search-client.version}</version>
        </dependency>

更新 1： 此处建议的 API 密钥的 Base64 编码（参见下面的代码）没有帮助。

            RequestOptions.Builder builder = RequestOptions.DEFAULT.toBuilder();
            builder.addHeader("Authorization", String.format("ApiKey %s",
                    Base64.getEncoder().encodeToString(API_KEY.getBytes(StandardCharsets.UTF_8))));
            final RequestOptions requestOptions = builder.build();

更新 2：改变我创建客户端的方式，也没有帮助（见下文）。

            Header[] defaultHeaders =
                    new Header[]{new BasicHeader("Authorization",
                            String.format("ApiKey %s",API_KEY))};
            final RestClientBuilder builder1 = RestClient.builder(CLOUD_ID);
            builder1.setDefaultHeaders(defaultHeaders);
            client = new RestHighLevelClient(
                    builder1
            );

更新 3：我将提供的 API 密钥更改为

public static final String BASE64_API_KEY = Base64.getEncoder().encodeToString(String.format("%s:%s", ID, KEY).getBytes());

正如里卡多费雷拉所建议的那样。

现在我得到一个不同的错误：

org.elasticsearch.client.ResponseException: method [PUT], host [XXXXXXXXXXXXXXXX], URI [/posts/_doc/1?timeout=1m], status line [HTTP/1.1 403 Forbidden]
{"error":{"root_cause":[{"type":"security_exception","reason":"action [indices:admin/auto_create] is unauthorized for API key id [XXXXXXXXXXXXXXXX] of user [XXXXXXXXXXXXXXXX]"}],"type":"security_exception","reason":"action [indices:admin/auto_create] is unauthorized for API key id [XXXXXXXXXXXXXXXX] of user [XXXXXXXXXXXXXXXX]"},"status":403}
        at org.elasticsearch.client.RestClient.convertResponse(RestClient.java:326)
        at org.elasticsearch.client.RestClient.performRequest(RestClient.java:296)
        at org.elasticsearch.client.RestClient.performRequest(RestClient.java:270)
        at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1621)
        ... 30 more

更新 4：

在我创建有问题的索引后，错误消息变为：

org.elasticsearch.client.ResponseException: method [PUT], host [XXXXXXXXXXXXXXXX], URI [/camunda-1/_doc/1?timeout=1m], status line [HTTP/1.1 403 Forbidden]
{"error":{"root_cause":[{"type":"security_exception","reason":"action [indices:data/write/bulk[s]] is unauthorized for API key id [XXXXXXXXXXXXXXXX] of user [XXXXXXXXXXXXXXXX]"}],"type":"security_exception","reason":"action [indices:data/write/bulk[s]] is unauthorized for API key id [XXXXXXXXXXXXXXXX] of user [XXXXXXXXXXXXXXXX]"},"status":403}

Answer 1

它不起作用，因为您使用了错误的 API 密钥。

但别担心：这些事情经常发生。 这肯定发生在我身上。

The API Key that you are creating is for you to issue REST requests against Elasticsearch Service — which is the entity that governs your Elasticsearch and Kibana clusters.

要使其工作，您需要专门从 Elasticsearch 创建一个 API 密钥。 要创建一个 go 到Dev Tools Console并发出以下请求：

POST _security/api_key
{
   "name": "my-api-key",
   "expiration": "7d", 
   "role_descriptors": {
      "custom-role": {
         "cluster": ["all"],
         "index": [
            {
               "names": [
                  "index-1",
                  "index-2"
               ],
               "privileges": ["all"]
            }
         ]
      }
   }
}

如果执行成功，您将得到如下响应：

{
  "id" : "liKs_XcBrNsSAgwboCN9",
  "name" : "my-api-key",
  "expiration" : 1615473484899,
  "api_key" : "NC3ZeIb_SGWjGJRZVoOf2g"
}

记下字段id和api_key 。 您将需要它们来创建授权 header：

String apiKey = String.format("%s:%s", id, api_key);
apiKey = Base64.getEncoder().encodeToString(apiKey.getBytes());
String authorization = String.format("ApiKey %s", apiKey);

之后，只需使用 Java 代码中的授权：

builder.addHeader("Authorization", authorization);

玩得开心