How to optimal when run multiple deep learning model on GPU

Question

I'm building the system that handle five streams concurrently. I'm facing the problem when I run multiple deep learning model in order to handle those streams then speed down very much. So, if you are used to design the system as so. Please comment to me some suggests.

Answer 1

I would consider to run multiple deep learning models each on a separate machine. Otherwise, you will always have them fighting for shared resources like RAM, CPU time, HDD, etc, and you won't achieve optimal performance.