简体   繁体   English

核大小应该与一维卷积中的字大小相同吗?

[英]Should Kernel size be same as word size in 1D Convolution?

In CNN literature, it is often illustrated that kernel size is same as size of the longest word in the vocabulary list that one has, when it sweeps across a sentence.在 CNN 文献中,经常说明核大小与词汇表中最长单词的大小相同,当它扫过一个句子时。

So if we use embedding to represent the text, then shouldn't the kernel size be same as the embedding dimension so that it gives the same effect as sweeping word by word?那么如果我们使用embedding来表示文本,那么内核大小不应该和embedding维度一样,这样就可以得到和逐词扫描一样的效果吗?

I see difference sizes of kernel used, despite the word length.尽管字长不同,但我看到使用的内核大小不同。

在此处输入图片说明

Well... these are 1D convolutions, for which the kernels are 3 dimensional.嗯……这些是一维卷积,其内核是 3 维的。

It's true that one of these 3 dimensions must match the embedding size (otherwise it would be pointless to have this size)确实,这 3 个维度之一必须与嵌入大小匹配(否则拥有此大小将毫无意义)

These three dimensions are:这三个维度是:

(length_or_size, input_channels, output_channels)  

Where:在哪里:

  • length_or_size ( kernel_size ): anything you want. length_or_sizekernel_size ):你想要的任何东西。 In the picture, there are 6 different filters with sizes 4, 4, 3, 3, 2, 2, represented by the "vertical" dimension.图中有6个不同的过滤器,大小分别为4、4、3、3、2、2,用“垂直”维度表示。
  • input_channels (automatically the embedding_size ): the size of the embedding - this is somwehat mandatory (in Keras this is automatic and almost invisible), otherwise the multiplications wouldn't use the entire embedding, which is pointless. input_channels (自动为embedding_size ): embedding_size的大小 - 这是一些强制性的(在 Keras 中这是自动的并且几乎不可见),否则乘法不会使用整个嵌入,这是毫无意义的。 In the picture, the "horizontal" dimension of the filters is constantly 5 (the same as the word size - this is not a spatial dimension).在图中,过滤器的“水平”维度始终为5(与字大小相同——这不是空间维度)。
  • output_channels ( filters ): anything you want, but it seems the picture is talking about 1 channel only per filter, since it's totally ignored, and if represented would be something like "depth". output_channels ( filters ):任何你想要的东西,但似乎图片只讨论了每个过滤器的 1 个通道,因为它被完全忽略了,如果代表的话会像“深度”这样的东西。

So, you're probably confusing which dimensions are which.因此,您可能会混淆哪些维度是哪些。 When you define a conv layer, you do:定义 conv 层时,您会执行以下操作:

Conv1D(filters = output_channels, kernel_size=length_or_size)

While the input_channels come from the embedding (or the previous layer) automatically.input_channels自动来自嵌入(或前一层)。

Creating this model in Keras在 Keras 中创建此模型

To create this model, it would be something like:要创建此模型,它将类似于:

sentence_length = 7
embedding_size=5

inputs = Input((sentence_length,))
out = Embedding(total_words_in_dic, embedding_size)

Now, supposing these filters have 1 channel only (since the image doesn't seem to consider their depth...), we can join them in pairs of 2 channels:现在,假设这些过滤器只有 1 个通道(因为图像似乎没有考虑它们的深度......),我们可以将它们成对地加入 2 个通道:

size1 = 4
size2 = 3
size3 = 2
output_channels=2

out1 = Conv1D(output_channels, size1, activation=activation_function)(out)
out2 = Conv1D(output_channels, size2, activation=activation_function)(out)
out3 = Conv1D(output_channels, size3, activation=activation_function)(out)

Now, let's collapse the spatial dimensions and remain with the two channels:现在,让我们折叠空间维度并保留两个通道:

out1 = GlobalMaxPooling1D()(out1)
out2 = GlobalMaxPooling1D()(out2)
out3 = GlobalMaxPooling1D()(out3)

And create the 6 channel output:并创建 6 通道输出:

out = Concatenate()([out1,out2,out3])

Now there is a mistery jump from 6 channels to 2 channels which cannot be explained by the picture.现在有一个从 6 通道到 2 通道的神秘跳跃,这无法用图片来解释。 Perhaps they're applying a Dense layer or something.......也许他们正在应用 Dense 层或其他东西......

#????????????????
out = Dense(2, activation='softmax')(out)

model = Model(inputs, out)

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 为什么不能在一维卷积中设置内核大小? - why I can't set kernel size in 1d convolution? Keras 1d 卷积层如何处理词嵌入 - 文本分类问题? (过滤器、内核大小和所有超参数) - How does Keras 1d convolution layer work with word embeddings - text classification problem? (Filters, kernel size, and all hyperparameter) Keras 中的 1D CNN:如果过滤器的数量和 kernel_size 太低,它会在序列中间停止卷积吗? - 1D CNN in Keras: if the number of filters and kernel_size are too low, will it stop convolution at the middle of a sequence? 在keras中使用2d内核执行1d卷积 - Performing 1d convolution using 2d kernel in keras keras 卷积层中的内核大小是如何定义的? - How is kernel size in keras convolution layers defined? 喀拉拉邦的一维卷积序列 - 1D convolution sequence in keras 'conv2d_2/convolution' 从 1 中减去 3 导致的负维度大小 - Negative dimension size caused by subtracting 3 from 1 for 'conv2d_2/convolution' 无法将1D信号前馈给Keras中的1D卷积 - Fail to feed forward 1D signal to 1D convolution in Keras keras 中一维卷积网络的输入形状 - Input shape for 1D convolution network in keras Keras - 一维卷积它是如何工作的 - Keras - 1D Convolution How it works
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM