PyTorch：是否可以将模型存储在CPU ram中，但是对于大型模型，可以在GPU上运行所有操作？

Question

From what I see, most people seem to be initializing an entire model, and sending the whole thing to the GPU. But I have a neural net model that is too big to fit entirely on my GPU. Is it possible to keep the model saved in ram, but run all the operations on the GPU?

Answer 1

I do not believe this is possible. However, one easy work around would be to split you model into sections that will fit into gpu memory along with your batch input.

1. Send the first part(s) of the model to gpu and calculate outputs
2. Release the former part of the model from gpu memory, and send the next section of the model to the gpu.
3. Input the output from 1 into the next section of the model and save outputs.

Repeat 1 through 3 until you reach your models final output.