Elasticsearch中缺少的文档

Question

I am using elasticsearch 0.90.10 . Using mongodb river I indexed the documents from mongodb. The initial load and indexing are working fine. But after some time the count of documents starts decreasing gradually (losing the documents). I am using mongodb-river version 1.7.4

These are the configs in my elasticsearch.yml.

Cluster Properties

cluster.name: 192.168.31.50:9300
index.store.type: niofs
node.max_local_storage_nodes: 1
transport.tcp.port: 9300
http.port: 9200
path.logs: /home/test/Test/scripts/../logs/search
discovery.zen.ping.multicast.enabled: false

Search pool

threadpool.search.type: fixed
threadpool.search.size: 5
threadpool.search.queue_size: 200

Bulk pool

threadpool.bulk.type: fixed
threadpool.bulk.size: 5
threadpool.bulk.queue_size: 300

Index pool

threadpool.index.type: fixed
threadpool.index.size: 5
threadpool.index.queue_size: 200

Indices settings

indices.memory.index_buffer_size: 30%
indices.memory.min_shard_index_buffer_size: 12mb
indices.memory.min_index_buffer_size: 96mb

Cache Sizes

indices.fielddata.cache.size: 15%
indices.fielddata.cache.expire: 6h
indices.cache.filter.size: 15%
indices.cache.filter.expire: 6h

Indexing Settings for Writes

index.refresh_interval: 30s
index.translog.flush_threshold_ops: 50000

These configurations are causing document loss in my production environment. What can I do to pinpoint the exact problem?

Thanks in advance!

Answer 1

Is it possible the ttl property being set? That's the only thing I've ever heard of "removing" documents.

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-ttl-field.html