[英]Should Kernel size be same as word size in 1D Convolution?
In CNN literature, it is often illustrated that kernel size is same as size of the longest word in the vocabulary list that one has, when it sweeps across a sentence.在 CNN 文献中,经常说明核大小与词汇表中最长单词的大小相同,当它扫过一个句子时。
So if we use embedding to represent the text, then shouldn't the kernel size be same as the embedding dimension so that it gives the same effect as sweeping word by word?那么如果我们使用embedding来表示文本,那么内核大小不应该和embedding维度一样,这样就可以得到和逐词扫描一样的效果吗?
I see difference sizes of kernel used, despite the word length.尽管字长不同,但我看到使用的内核大小不同。
Well... these are 1D convolutions, for which the kernels are 3 dimensional.嗯……这些是一维卷积,其内核是 3 维的。
It's true that one of these 3 dimensions must match the embedding size (otherwise it would be pointless to have this size)确实,这 3 个维度之一必须与嵌入大小匹配(否则拥有此大小将毫无意义)
These three dimensions are:这三个维度是:
(length_or_size, input_channels, output_channels)
Where:在哪里:
length_or_size
( kernel_size
): anything you want. length_or_size
( kernel_size
):你想要的任何东西。 In the picture, there are 6 different filters with sizes 4, 4, 3, 3, 2, 2, represented by the "vertical" dimension.input_channels
(automatically the embedding_size
): the size of the embedding - this is somwehat mandatory (in Keras this is automatic and almost invisible), otherwise the multiplications wouldn't use the entire embedding, which is pointless. input_channels
(自动为embedding_size
): embedding_size
的大小 - 这是一些强制性的(在 Keras 中这是自动的并且几乎不可见),否则乘法不会使用整个嵌入,这是毫无意义的。 In the picture, the "horizontal" dimension of the filters is constantly 5 (the same as the word size - this is not a spatial dimension).output_channels
( filters
): anything you want, but it seems the picture is talking about 1 channel only per filter, since it's totally ignored, and if represented would be something like "depth". output_channels
( filters
):任何你想要的东西,但似乎图片只讨论了每个过滤器的 1 个通道,因为它被完全忽略了,如果代表的话会像“深度”这样的东西。 So, you're probably confusing which dimensions are which.因此,您可能会混淆哪些维度是哪些。 When you define a conv layer, you do:
定义 conv 层时,您会执行以下操作:
Conv1D(filters = output_channels, kernel_size=length_or_size)
While the input_channels
come from the embedding (or the previous layer) automatically.而
input_channels
自动来自嵌入(或前一层)。
To create this model, it would be something like:要创建此模型,它将类似于:
sentence_length = 7
embedding_size=5
inputs = Input((sentence_length,))
out = Embedding(total_words_in_dic, embedding_size)
Now, supposing these filters have 1 channel only (since the image doesn't seem to consider their depth...), we can join them in pairs of 2 channels:现在,假设这些过滤器只有 1 个通道(因为图像似乎没有考虑它们的深度......),我们可以将它们成对地加入 2 个通道:
size1 = 4
size2 = 3
size3 = 2
output_channels=2
out1 = Conv1D(output_channels, size1, activation=activation_function)(out)
out2 = Conv1D(output_channels, size2, activation=activation_function)(out)
out3 = Conv1D(output_channels, size3, activation=activation_function)(out)
Now, let's collapse the spatial dimensions and remain with the two channels:现在,让我们折叠空间维度并保留两个通道:
out1 = GlobalMaxPooling1D()(out1)
out2 = GlobalMaxPooling1D()(out2)
out3 = GlobalMaxPooling1D()(out3)
And create the 6 channel output:并创建 6 通道输出:
out = Concatenate()([out1,out2,out3])
Now there is a mistery jump from 6 channels to 2 channels which cannot be explained by the picture.现在有一个从 6 通道到 2 通道的神秘跳跃,这无法用图片来解释。 Perhaps they're applying a Dense layer or something.......
也许他们正在应用 Dense 层或其他东西......
#????????????????
out = Dense(2, activation='softmax')(out)
model = Model(inputs, out)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.