Watson Assistant中的逻辑确定对话框

Question

I want to improve ibm's Watson assistant results. So, I want to know the algorithm to determine a dialog in Watson assistant's conversations. Is it a svm algorithm? A paper is welcome.

Answer 1

There are a number of ML/NLP technologies under the covers of Watson Assistant. So it's not just a single algorithm. Knowing them is not going to help you improve your results.

I want to improve ibm's Watson assistant results.

There are a number of ways.

Representative questions.

Focus on getting true representative questions from the end users. Not only in the language that they use, but if possible from the same medium you plan to use WA on (eg. Mobile device, Web, Audio).

This is the first factor that reduces accuracy. Manufacturing an intent can mean you build an intent that a customer may never ask (even if you think they do). Second you will use language/terms with similar patterns. This makes it harder for WA to train.

Total training questions

It's possible to train an intent with one question, but for best results 10-20 example questions. Where intents are close together then more examples are needed.

Testing

The current process is to create what is called a K-Fold Cross validation ( sample script ). If your questions are representative then the results should give you an accurate indicator of how well it is performing.

However, it is possible to overfit the training. So you should use a blind set. This is a 10-20% of all questions (Random sample). They should never be used to train WA. Then run them against the system. Both your Blind + K-Fold should fall within 5% of each other.

You can look at the results of the K-Fold to fix issues, but blind set you should not. Blinds can go stale as well. So try to create a new blind set after 2-3 training cycles.

End user testing.

No matter how well your system is trained, I can guarantee you that new things will pop up when put in front of end users. So you should plan to have users test before you put it into production.

When getting users to test, ensure they understand the general areas it has been trained on. You can do this with user stories, but try not to prime the user into asking a narrow scoped question.

Example:

• "Your phone is not working and you need to get it fixed" - Good. They will ask questions you will never have seen before.
• "The wifi on your phone is not working. Ask how you would fix it". - Bad. Very narrow scope and people will mention "wifi" even if they don't know what it means.