如何获取 AWS EMR 的实例列表？

Question

为什么 EC2 的列表与 EMR 列表不同？

EC2： https : //aws.amazon.com/ec2/spot/pricing/

EMR： https : //aws.amazon.com/emr/pricing/

为什么不是所有类型的 EC2 实例都可用于 EMR？ 如何获得这份特别名单？

Answer 1

如果您的问题与亚马逊控制台无关
（那么它肯定会被关闭为题外话）：

作为编程解决方案，您看起来像这样：（使用 python boto3 ）

import boto3
client = boto3.client('emr')
for instance in client.list_instances():
  print("Instance[%s] %s"%(instance.id, instance.name))

Answer 2

这就是我使用的，虽然我不是 100% 确定它是准确的（因为我找不到文档来支持我的一些选择（-BoxUsage 等））。

值得查看来自 AWS 的响应，以找出定价客户端响应中不同字段的不同值。

使用以下命令获取响应列表：

   default_profile = boto3.session.Session(profile_name='default')
    # Only us-east-1 has the pricing API
    # - https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/pricing.html
    pricing_client = default_profile.client('pricing', region_name='us-east-1')
    service_name = 'ElasticMapReduce'
    product_filters = [
        {'Type': 'TERM_MATCH', 'Field': 'location', 'Value': aws_region_name}
    ]
    response = pricing_client.get_products(
        ServiceCode=service_name,
        Filters=product_filters,
        MaxResults=100
    )
    response_list.append(response)

    num_prices = 100
    while 'NextToken' in response:
       # re-query to get next page

获得响应列表后，您可以过滤掉实际的实例信息：

  emr_prices = {}
  for response in response_list:
      for price_info_str in response['PriceList']:
          price_obj = json.loads(price_info_str)
          attributes = price_obj['product']['attributes']

          # Skip pricing info that doesn't specify a (EC2) instance type
          if 'instanceType' not in attributes:
              continue
          inst_type = attributes['instanceType']

          # AFAIK, Only usagetype attributes that contain the string '-BoxUsage' are the ones that contain the prices that we would use (empirical research)
          # Other examples of values are <REGION-CODE>-M3BoxUsage, <REGION-CODE>-M5BoxUsage, <REGION-CODE>-M7BoxUsage (no clue what that means.. )
          if '-BoxUsage' not in attributes['usagetype']:
              continue

          if 'OnDemand' not in price_obj['terms']:
              continue

          on_demand_info = price_obj['terms']['OnDemand']
          price_dim = list(list(on_demand_info.values())[0]['priceDimensions'].values())[0]
          emr_price = Decimal(price_dim['pricePerUnit']['USD'])

          emr_prices[inst_type] = emr_price

实际上，从 boto3 文档中弄清楚这一点很简单。 特别是get_products文档。