通过Bluemix Retrieve＆Rank，我们如何实现持续学习的系统？

Question

With reference to the Web page below, using the Retrieve & Rank service of IBM Bluemix, we are creating a bot that can respond to inquiries.

Question: After learning the ranker once, based on the user's response to the inquiry, how can we construct a mechanism to continuously learn and improve response accuracy?

Assumption: Because there was no API of R&R service to continuously learn from the inquiry response result of the user, tuning the GroundTruth file, I suppose that it is necessary to periodically perform such a process as training the ranker again.

Tuning contents of assumed GT file:

• If there is a new question, add a set of questions and answers
• Increase or decrease the relevance score of responses if there is something that could not be answered well by existing question (If bot answered incorrectly, lower the score, if there is a useful answer, raise the score)

Answer 1

In order to continuously learn, you will want to do the following:

• capture new examples ie each user input and corresponding result
• review those examples and create new ranker examples, adjust relevance scores, etc
• add those new examples to the ranker
• retrain the ranker using your new and existing examples

NOTE: Be sure to validate that new updates to the ranker data improves the overall system performance. k-fold validation is a great way to measure this.

All in all, learning is a continuous process that should be repeated indefinitely or until system performance is deemed suffcient.