Python InfluxDB2 - write_api.write(...) How to check for success?

Question

I need to write historic data into InfluxDB (I'm using Python, which is not a must in this case, so I maybe willing to accept non-Python solutions). I set up the write API like this

write_api = client.write_api(write_options=ASYNCHRONOUS)

The Data comes from a DataFrame with a timestamp as key, so I write it to the database like this

result = write_api.write(bucket=bucket, data_frame_measurement_name=field_key, record=a_data_frame)

This call does not throw an exception, even if the InfluxDB server is down. result has a protected attribute _success that is a boolean in debugging, but I cannot access it from the code.

How do I check if the write was a success?

Answer 1

if you want to immediately write data into database, then use SYNCHRONOUS version of write_api - https://github.com/influxdata/influxdb-client-python/blob/58343322678dd20c642fdf9d0a9b68bc2c09add9/examples/example.py#L12

The asynchronous write should be "triggered" by call .get() - https://github.com/influxdata/influxdb-client-python#asynchronous-client

Regards

Answer 2

write_api.write() returns a multiprocessing.pool.AsyncResult or multiprocessing.pool.AsyncResult (both are the same).

With this return object you can check on the asynchronous request in a couple of ways. See here: https://docs.python.org/2/library/multiprocessing.html#multiprocessing.pool.AsyncResult

If you can use a blocking request, then write_api = client.write_api(write_options=SYNCRONOUS) can be used.

Answer 3

If you use background batching, you can add custom success, error and retry callbacks.

from influxdb_client import InfluxDBClient

def success_cb(details, data):
    url, token, org = details
    print(url, token, org)
    print(data.decode('utf-8').split('\n'))
    print('Total Rows Inserted:', len(data.decode('utf-8').split('\n')))  

def error_cb(details, data, exception):
    print(exc)

def retry_cb(details, data, exception):
    print('Retrying because of an exception:', exc)    


with InfluxDBClient(url, token, org) as client:
    with client.write_api(success_callback=success_cb,
                          error_callback=error_cb,
                          retry_callback=retry_cb) as write_api:

        write_api.write(...)

If you are eager to test all the callbacks and don't want to wait until all retries are finished, you can override the interval and number of retries.

from influxdb_client import InfluxDBClient, WriteOptions

with InfluxDBClient(url, token, org) as client:
    with client.write_api(success_callback=success_cb,
                          error_callback=error_cb,
                          retry_callback=retry_cb,
                          write_options=WriteOptions(retry_interval=60,
                                                     max_retries=2),
                          ) as write_api:
        ...