How to continuously train our pre-trained model on real time data?

Question

I have some sensors which fetch data from cement factory and sends data to AWS IoT. The data is then tested on pre-trained model and the model predicts quality of cement based on some parameters. The data is coming in one second interval.

Since the data is coming in real-time, I want to train the model incrementally in real time.

Can anybody suggest how train model continuously?

Answer 1

You could aggregate certain numbers of training data and then use .partial_fit() to update your model.

.partial_fit() is the incremental learning option, which is available in Sklearn.

If your incremental data would not fit in RAM, then its worth trying dask-ml wrapper for incremental learning .