如何在Elasticsearch中有效存储文本内容并使之可搜索

Question

I have the data in the file in this form, how to split the different section and store it in Elasticsearch index and search based on some unique number.

Sample data:

SSLEGGU00402-IM    13949 13949    58     1  285228   3094844 1U00402-IM    13949
   200  1490   400  1490   600  1490   800  1490  1000  1490 2U00402-IM    13949
  1200  1490  1400  1491  1600  1493  1800  1497  2000  1504 3U00402-IM    13949
SSLEGGU00412-IM    13885 13885    58     1  286359   3094844 1U00412-IM    13885
   200  1489   400  1489   600  1489   800  1489  1000  1489 2U00412-IM    13885
  1200  1489  1400  1490  1600  1493  1800  1497  2000  1505 3U00412-IM    13885

I would like to store SSLEGGU00402 as a separate document and SSLEGGU00412 as a separate document and i need to search based on the same.

Does Elasticsearch by-default gives some way to split this text and store it or we need to split it programmatically and store as Elasticsearch Index.

Answer 1

A good start would be to look into elasticsearch 's Ingest Node and its Processors . They can be used to apply transformations on documents before they get indexed.

If your source data is well-defined and adheres to a specific pattern, I hope you can make use of GROK processor to convert it to a structured JSON for indexing.

https://www.elastic.co/guide/en/elasticsearch/reference/current/grok-processor.html

In case if you have sophisticated logic that needs to be applied for data pre-processing, you can make your own pipeline

https://www.elastic.co/guide/en/elasticsearch/reference/6.2/ingest.html

Thanks.