Loki 存储在 AWS S3 上的问题

Question

根据标题：我对 Loki（在 Docker 上运行）存储其块和 C 有疑问。在 AWS S3 的存储桶上。 Loki 运行良好，只是将其日志存储在文件系统中而不是存储桶中，事实上存储桶是空的。 我的配置有什么问题？

在 AWS IAM 中，我创建了一个具有编程访问权限的用户，并为其提供了以下策略......

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "LokiStorage",
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::__myBucket__",
                "arn:aws:s3:::__myBucket__/*"
            ]
        }
    ]
}

该策略似乎足够了，因为我已经使用它来将文件系统中存在的一些文件推送到存储桶中。 docker的相关部分组成...

version: "3.8"
volumes:
    loki_data: {}
services:    
  loki:
    image: grafana/loki:2.1.0
    networks:
      - my-overlay
    ports:
      - 3100:3100
    volumes:
      - ./loki/loki-config.yml:/etc/loki/local-config.yml
      - loki_data:/loki
    command: -config.file=/etc/loki/local-config.yaml

...似乎也很好：Loki - 作为容器和服务 - 运行平稳。 Loki 的配置文件“loki-config.yaml”似乎也不错...

---
auth_enabled: false
ingester:
  chunk_idle_period: 3m
  chunk_block_size: 262144
  chunk_retain_period: 1m
  max_transfer_retries: 0
  lifecycler:
    ring:
      kvstore:
        store: inmemory
      replication_factor: 1
limits_config:
  enforce_metric_name: false
  reject_old_samples: true
  reject_old_samples_max_age: 168h
compactor:
  working_directory: /loki/boltdb-shipper-compactor
  shared_store: aws
schema_config:
  configs:
    - from: 2020-07-01
      store: boltdb-shipper
      object_store: aws
      schema: v11
      index:
        prefix: loki_index_
        period: 24h
server:
  http_listen_port: 3100
storage_config:
  aws:
    s3: s3://__myAccessKey__:__mySecretAccessKey__@eu-west-1/__myBucket__
  boltdb_shipper:
    active_index_directory: /loki/index
    shared_store: s3
    cache_location: /loki/boltdb-cache
  chunk_store_config:
    max_look_back_period: 0s
  table_manager:
    retention_deletes_enabled: false
    retention_period: 0s

... 但是，事实上，不知何故，某处有问题和/或缺失。 建议？

Answer 1

我找到了罪魁祸首：docker-compose 出错了：

volumes: 
  - ./loki/loki-config.yml:/etc/loki/local-config.yml
### Here: "yml", and not "yaml" ──────────────────┘
  - loki_data:/loki
command: -config.file=/etc/loki/local-config.yaml
### Here: "yaml", and not "yml" ─────────────┘

现在编写正确，Loki的配置文件如下：

auth_enabled: false
server:
  http_listen_port: 3100
ingester:
  lifecycler:
    address: 127.0.0.1
    ring:
      kvstore:
        store: inmemory
      replication_factor: 1
    final_sleep: 0s
  chunk_idle_period: 1h
  max_chunk_age: 1h    
  chunk_target_size: 1048576
  chunk_retain_period: 30s  
  max_transfer_retries: 0   
schema_config:
  configs:
    - from: 2020-10-24
      store: boltdb-shipper
      object_store: s3
      schema: v11
      index:
        prefix: index_
        period: 24h
storage_config:
  boltdb_shipper:
    active_index_directory: /loki/index
    cache_location: /loki/index_cache
    cache_ttl: 24h         # Can be increased for faster performance over longer query periods, uses more disk space
    shared_store: s3
  aws:
    s3forcepathstyle: true
    bucketnames: __myBucket__
    region: eu-west-1
    access_key_id: __myAccessKey__
    secret_access_key: __mySecretAccessKey__
compactor:
  working_directory: /loki/compactor
  shared_store: s3
  compaction_interval: 5m
limits_config:
  reject_old_samples: true
  reject_old_samples_max_age: 168h
  ingestion_rate_mb: 16
  ingestion_burst_size_mb: 32
chunk_store_config:
  max_look_back_period: 0s
table_manager:
  retention_deletes_enabled: false
  retention_period: 0s
ruler:
  storage:
    type: s3
    s3:
      s3forcepathstyle: true
      bucketnames: __myBucket__
      region: eu-west-1
      access_key_id: __myAccessKey__
      secret_access_key: __mySecretAccessKey__
  rule_path: /loki/rules-temp
  alertmanager_url: http://localhost:9093
  ring:
    kvstore:
      store: inmemory
  enable_api: true

配置似乎很好，因为 - 最后。 - S3 的存储桶已填充且 Loki 工作正常。

或许。

奇怪的是：存储桶（如上所述）已填充，但我看到 Loki 还在本地文件系统中存储了它的块等：

root@ip-aaa-bbb-ccc-ddd:/var/lib/docker/volumes/monit_loki_data# tree
.
└── _data
    ├── compactor
    │   ├── index_19375
    │   └── index_19376
    ├── index
    │   └── uploader
    │       └── name
    ├── index_cache
    │   └── index_19376
    │       ├── 5a22562e87b2-1674059149901907563-1674116100.gz
    │       └── compactor-1674116450.gz
    ├── rules
    └── tmprules

10 directories, 3 files

因此问题来了：Loki 真的按预期工作吗？ 那就是：在存储桶和文件系统中存储它是它的正常行为吗？