简体繁体 English

Postgres到ELK Stack的ElasticSearch数据索引

[英]Postgres to ElasticSearch data indexing for ELK Stack

原文 2017-03-01 20:15:46 4 1 postgresql/ elasticsearch/ elastic-stack

So I know the JDBC Rivers plugin is deprecated so even though it is being used I'd ideally not want to look at using something that is no longer supported. 因此，我知道JDBC Rivers插件已被弃用，因此即使正在使用它，我理想上也不想使用不再受支持的插件。

However I have a few tables in a Postgres database with values that I need to be able to search in a Kibana view. 但是，我在Postgres数据库中有一些表，这些表需要能够在Kibana视图中进行搜索的值。 Im new to the ELK stack but i've been messing around with some of their samples to get familiar. 我是ELK堆栈的新手，但是我一直在弄弄他们的一些示例以熟悉它们。

I've seen some mentions of using Stored Procedures/Triggers from Postgres to send to Logstash. 我已经看到一些使用Postgres中的存储过程/触发器将其发送到Logstash的提及。 Although im not sure if this is the best way. 虽然我不确定这是否是最好的方法。 Im not a developer but a QA so my coding skills are "ok" as im used to writing automation tests/etc... 我不是开发人员，而是质量检查人员，因此我的编码技能“不错”，因为我习惯于编写自动化测试/等。

What would be the best way to do this? 最好的方法是什么？ I would want to probably capture updates to these tables (probably new inserts or updates) OR be able to poll the data every X period of time (30s or something). 我可能想捕获这些表的更新（可能是新的插入或更新），或者能够每隔X个时间段（30秒左右）轮询一次数据。 Lets pretend it's for a weather station and the tables contain humidity data from different weather sensors. 假设它是一个气象站，并且表格中包含来自不同气象传感器的湿度数据。

I'd want to be able to search in a Kibana view the Values/Station ID/etc... 我希望能够在Kibana中搜索值/站点ID /等...

Is this doable? 这可行吗？ Is there maybe a better way than using Triggers/Stored procedures? 是否有比使用触发器/存储过程更好的方法？

1 个解决方案

I ended up using the JDBC driver and following https://www.elastic.co/blog/logstash-jdbc-input-plugin to get it moving and working (Which it does move). 我最终使用了JDBC驱动程序，并按照https://www.elastic.co/blog/logstash-jdbc-input-plugin进行了移动和运行（确实移动了）。 But it was a lot of setup for anyone that may see this answer. 但这对可能看到此答案的任何人来说都是很多设置。