How to get data from Oracle to send it to statsd or directly into InfluxDB?

Question

Is there a solution to get some data from Oracle database for further send it to statsd or directly into InfluxDB? I have a lot of sql queries that I need to run periodically for getting some counters. I need an alternative of ORABBIX (zabbix), that have a persistent connection to database, but for stastd/InfluxDB. I want to reduce an connections to database while querying for counters that is in tables. Thank you.

Answer 1

Since there is the cx_Oracle and InfluxDBClient you can write a simple Python script that opens all neccessary DB connections, prepares a bunch of SQL statements and repeatedly executes them and feeds them into a influxdb instance.

Basically something along those lines:

from influxdb import InfluxDBClient
import cx_Oracle
ic = InfluxDBClient('vigilante.example.org', 8086, 'collector', 'pw', 'db'

db_dict = {}
for sid in sids:
  db_dict[sid] = cx_Oracle.connect(ORACLE_USER, ORACLE_PASS, sid)

cursor_dict = {}
for (name, c) in config.items():
  cursor = db_dict[c['sid']].cursor()
  cursor.prepare(c['config'])
  cursor_dict[name, c['sid']] = (cursor, c['line'])

def collect(cursor_dict):
  vs = []
  for ((name, sid), (cursor, line)) in cursor_dict:
    rows.execute(cursosr.fetchall())
    if rows:
      vs.append(line.format(sid, *rows[0]))
  if vs:
    ic.write_points(vs, protocol='line')

while True:
  collect(cursor_dict)
  time.sleep(sleep_for_some_dynamically_computed_time)

In that example, each SQL statement returns exactly one row and has a influxdb line associated ( c['line'] ) which looks like:

measurement,db={} col_name1={},col_name2={},col_name3={}

You can also add a timestamp to that line - if it is omitted, influxdb uses the current time. Note that this script is long running - and uses prepared statements to avoid wasting DB resources.

Alternatively, if you already have a collectd running: there is an oracle plugin for collectd. Thus, you could use your existing queries with that plugin and configure collectd to forward the data points to an influxdb instance (via UDP, ie enable a UDP listening port in the influxdb config).

Answer 2

To sent the data directly into the influxDB:

We can write a java code to insert the data into the influxDB using influxdb-java API

We can create a oracle ojdbc connection, query the data using a sql query and store the result in a ResultSet and create a influxDB connection, read the queried data from the result and store it as point into the influxDB measurement.

We can add the time stamp while creating a point or ignore it , in that case it will automatically take the system date as a time stamp.

Refer below link for reference: https://github.com/influxdata/influxdb-java/blob/master/src/test/java/org/influxdb/InfluxDBTest.java