Can I make a transformer based chatbot which is pretrained on some other dataset?

Question

I'm currently making a medical transformer chatbot from the tutorial from this page:

https://blog.tensorflow.org/2019/05/transformer-chatbot-tutorial-with-tensorflow-2.html

I'm using a text corpus of medical questions/answers. When I train the model and test it, it just gives answers to questions relevant to medical domain. However, I want to create a conversational chatbot which can answer basic questions like 'How are you' and 'I need help'. Is there a way I can us some pretrained weights, then train the model on my medical dataset? I'm pretty new to natural language processing domain so could really use some guidance. Thanks!

Answer 1

General solution is to design some dialog template that you can fill with question answers. Then use the qa model to generate synthetic dataset. You should be able to use the generated data to train/finetune/setup your chatbot. The rest is about important details in a project you got there!