是否应该从 Rasa NLU 训练数据中删除标点符号？

Question

在 NLU 训练数据中，意图的话语中的标点符号（逗号、撇号、问号、大写字母等）是否应该保持原样、删除，还是根本不重要？

Answer 1

The training data can be left with punctuation and the WhitespaceTokenizer ( documentation link ) will clean it up. Not all punctuations are cleaned up though! You can see the regex used in the tokenizer on Github .

So for your mentioned punctuation like commas, apostrophes, question marks, etc. you can leave it there and the tokenizer will handle it.