Why read access takes time after restarting influxdb?

Question

So whenever we restart influxdb, we notice that it takes quite a while for grafana to reconnect/read data from influxdb.(up to ~10m)

However, when it comes to writing data and accessing /health endpoint the DB is available within ~2-3 mins.

Why does the DB take a longer time to serve reads after the DB service/process is restarted?

Is it related to WAL flushing , but why does that affect read?

Nothing indicated on logs either.

Influxdb: 1.7

Answer 1

Influx becomes available for writing so fast because it is able to load the WAL, and the in-memory database can then be populated as new data points are loaded. So on startup, very little needs to happen before the system is able to quickly process new writes. But even though all points in the WAL are flushed to permanent storage, the system needs to organize and perform compaction on the index files which is a costly operation, and must be completed before routine reads will perform as expected.

This was more of a problem in the early days of Influx when startup time could easily exceed an hour for larger DB's with more series' or longer keys. It has been tweaked over the years, so really you should consider 10 minutes as a blessing and not a sign of trouble.