如何使用 Elasticsearch JavaScript SDK 连接到 AWS Elasticsearch？

Question

我正在使用AWS Elasticsearch服务并希望通过elasticsearch.js进行连接，但需要一个端口。

看起来 AWS 只提供REST API （例如通过 curl），在端口 80 上运行。我的集群已启动，我可以通过浏览器访问，但不能通过 elasticsearch.js。

这个例子对我不起作用：

var elasticsearch = require('elasticsearch');
var client = new elasticsearch.Client({
  host: 'localhost:9200', // not working: '', 80, default: 443
  log: 'trace'
});

client.ping({
  requestTimeout: 1000
}, function (error) {
  if (error) {
    console.trace('elasticsearch cluster is down!');
  } else {
    console.log('All is well');
  }
});

我找到了http-aws-es ，但它也不起作用。

有什么想法吗？ 提前致谢！

Answer 1

对于elasticsearch.client，您可以将http-aws-es 用于connectionClass 和带有密钥的amazonES。

var client = new elasticsearch.Client({
    hosts: config.elasticsearch.host,
    connectionClass: require('http-aws-es'),
    amazonES: {
        region: config.aws.region,
        accessKey: config.aws.key,
        secretKey: config.aws.secret
    }
});

Answer 2

NPM 包elasticsearch已弃用，取而代之的是@elastic/elasticsearch

因此，而不是使用http-aws-es ，这是应该的工作使用旧版elasticsearch包，您可以考虑使用包@ acuris / AWS-ES-连接，为新elasticsearch客户的AWS ES连接@elastic/elasticsearch 。 它在我的项目中对我很有效。 你可以在它的自述文件中找到它的用法，但这里有一段示例代码：

import {
  createAWSConnection,
  awsGetCredentials,
} from '@acuris/aws-es-connection';
import { Client } from '@elastic/elasticsearch';

export const getESClient = async () => {
  const esEndpoint = process.env.AWS_ES_ENDPOINT;
  if (!esEndpoint) {
    throw new Error(
      'AWS_ES_ENDPOINT ENV not set.'
    );
  }

  const awsCredentials = await awsGetCredentials();
  const AWSConnection = createAWSConnection(awsCredentials);
  const client = new Client({
    ...AWSConnection,
    node: esEndpoint,
  });
  return client;
};

export const createNewIndex = async (index: string) => {
  try {
    const client = await getESClient();
    const exists = await client.indices.exists({ index });
    if (!exists || !exists.statusCode || exists.statusCode !== 404) {
      console.log(`Index ${index} might alrady exist.`, exists);
      return false;
    }
    const created = await client.indices.create({
      index,
      body: {
        mappings: {
          properties: {
            product_id: {
              type: 'keyword',
            },
            product_description: {
              type: 'text',
            },
          },
        },
      },
    });
    console.log(`Index created for ${index}`, created);
  } catch (error) {
    console.log(`Error creating index ${index}`, error);
    return false;
  }
  return true;
};

Answer 3

这是我在 TypeScript 中使用的Connection类的一个实现：

import { Connection as UnsignedConnection } from '@elastic/elasticsearch';
import * as AWS from 'aws-sdk';
import RequestSigner from 'aws-sdk/lib/signers/v4';
import { ClientRequest, IncomingMessage } from 'http';

class AwsElasticsearchError extends Error {}
type RequestOptions = Parameters<UnsignedConnection['request']>[0];


class AwsSignedConnection extends UnsignedConnection {
  public request(
    params: RequestOptions,
    callback: (err: Error | null, response: IncomingMessage | null) => void,
  ): ClientRequest {
    const signedParams = this.signParams(params);
    return super.request(signedParams, callback);
  }

  private signParams(params: RequestOptions): RequestOptions {
    const region = AWS.config.region || process.env.AWS_DEFAULT_REGION;
    if (!region) throw new AwsElasticsearchError('missing region configuration');
    if (!params.method) throw new AwsElasticsearchError('missing request method');
    if (!params.path) throw new AwsElasticsearchError('missing request path');
    if (!params.headers) throw new AwsElasticsearchError('missing request headers');

    const endpoint = new AWS.Endpoint(this.url.href);
    const request = new AWS.HttpRequest(endpoint, region);

    request.method = params.method;
    request.path = params.querystring
      ? `${params.path}/?${params.querystring}`
      : params.path;
    request.body = params.body;

    Object.entries(params.headers).forEach(([header, value]) => {
      if (value === undefined) return;
      if (typeof value === 'string') request.headers[header] = value;
      else if (typeof value === 'number') request.headers[header] = `${value}`;
      else request.headers[header] = value.join('; ');
    });
    request.headers.Host = endpoint.host;

    const signer = new RequestSigner(request, 'es');
    signer.addAuthorization(AWS.config.credentials, new Date());
    return request;
  }
}

export { AwsSignedConnection, UnsignedConnection, AwsElasticsearchError };

然后您可以仅在凭据可用时提供它，因此您可以使用它来指向本地（例如 Docker）Elasticsearch，而无需凭据：

import awsSdk from 'aws-sdk';
import elasticsearch from '@elastic/elasticsearch';
import { AwsSignedConnection, UnsignedConnection } from '../aws-es-connector';

client = new elasticsearch.Client({
  Connection: awsSdk.config.credentials ? AwsSignedConnection : UnsignedConnection,
  node: elasticsearchEndpoint,
});

Answer 4

我花了一个小时找到适合我的方法。 唯一有效的是：

var elasticsearch = require('elasticsearch');
var AWS = require('aws-sdk');
var connectionClass = require('http-aws-es');
AWS.config.update({
    credentials: new AWS.Credentials('accessKey','secret'),
    region: 'us-east-1'
  });

var client = new elasticsearch.Client({
    host: 'https://yourdomainurl.us-east-1.es.amazonaws.com',
    log: 'debug',
    connectionClass: connectionClass,
    amazonES: {
      credentials: new AWS.EnvironmentCredentials('AWS')
    }
  });

确保您为 IAM 用户添加了访问策略。 那么一切都应该正常工作。 希望这会帮助某人。

Answer 5

Elasticsearch Service 端点在 HTTPS（端口 443）或 HTTP（端口 80）上运行。

您可以将您的 localhost:9200 交换到此以进行简单的交换替换