如何在訓練后向神經網絡 model 添加更多神經元/過濾器？

Question

I'm interested in training both a CNN model and a simple linear feed forward model in PyTorch, and after training to add more filters -- to the CNN layers, & neurons -- to the linear model layers and the outputs (eg from binary分類到多類分類）。 通過添加它們，我的具體意思是保持訓練的權重不變，並將隨機初始化的權重添加到新的傳入權重。

這里有一個 CNN model 的例子，這里有一個簡單的線性前饋 model 的例子

Answer 1

這個有點棘手，需要slice （有關slice的更多信息，請參閱此答案，但它應該是直觀的）。 這也是切片技巧的答案。 解釋請看評論：

import torch
    
def expand(
    original: torch.nn.Module,
    *args,
    **kwargs
    # Add other arguments if needed, like different stride
    # They won't change weights shape, but may change behaviour
):
    new = type(original)(*args, **kwargs)

    new_weight_shape = torch.tensor(new.weight.shape)
    new_bias_shape = torch.tensor(new.bias.shape)

    original_weight_shape = torch.tensor(original.weight.shape)
    original_bias_shape = torch.tensor(original.bias.shape)
    # I assume bias and weight exist, if not, do some checks
    # Also quick check, that new layer is "larger" than original
    assert torch.all(new_weight_shape >= original_weight_shape)
    assert new_bias_shape >= original_bias_shape

    # All the weights will be inputted from top to bottom, bias 1D assumed
    new.bias.data[:original_bias_shape] = original.bias.data

    # Create slices 0:n for each dimension
    slicer = tuple([slice(0, dim) for dim in original_weight_shape])
    # And input the data
    new.weight.data[slicer] = original.weight.data

    return new


layer = torch.nn.Conv2d(in_channels=16, out_channels=32, kernel_size=3)

new = expand(layer, in_channels=32, out_channels=64, kernel_size=3)

這應該適用於任何層（具有weight和bias ，如果需要進行調整）。 使用這種方法，您可以重新創建神經網絡或使用 PyTorch 的apply （ 此處的文檔）

另請記住，您必須為“新層”顯式傳遞創建的*args和**kwargs ，這將輸入經過訓練的連接。