卷积神经网络（CNN）可以用于无监督学习中的特征提取吗？

Question

I'm doing a side project to learn AI with ANN, I thought of making an unsupervised model that extracts features of each frame on a video to compare them in the future and detect image repetitions.

My idea is to use a CNN to extract for each frame the features but I can't seem to make it work, as I am learning my intuition tells me that there is something I am just not understanding.

How can I create an unsupervised model that extracts features of an array of images?

This is what I got:

img = load_image_func(???) # this loads a video and return a reshaped ordered list of frames 

input_shape = (150, 150, 3)

# The model
model = Sequential()
model.add(Conv2D(32, (3, 3), activation='relu', padding='same', name='conv_1', input_shape=input_shape))
model.add(MaxPooling2D((2, 2), name='maxpool_1'))
model.add(Conv2D(64, (3, 3), activation='relu', padding='same', name='conv_2'))
model.add(MaxPooling2D((2, 2), name='maxpool_2'))
model.add(Conv2D(128, (3, 3), activation='relu', padding='same', name='conv_3'))
model.add(MaxPooling2D((2, 2), name='maxpool_3'))
model.add(Conv2D(128, (3, 3), activation='relu', padding='same', name='conv_4'))
model.add(MaxPooling2D((2, 2), name='maxpool_4'))
model.add(Flatten())
model.add(Dropout(0.5))
model.add(Dense(512, activation='relu', name='dense_1'))
model.add(Dense(128, activation='relu', name='dense_2'))
model.add(Dense(67500, activation='sigmoid', name='output'))

optimizer=keras.optimizers.Adam(learning_rate=0.001)

model.compile(loss = 'sparse_categorical_crossentropy', optimizer= optimizer, metrics=['accuracy'])

#model.summary()

model.fit(vidcap, vidcap, batch_size=64, epochs=20)

I have the feeling I should be training the model but as it is unsupervised I don't have train data.

Also, how many units should I put in the output layer as I don't how many features will be detected?

Thanks for your time

Answer 1

Indeed, the CNN model will extract several features of an Image (eg colors, shapes, edges, patterns, etc.)

However , what are you defining as Images Repetition? Are you looking for an algorithm that finds similar images? If this is the case, then You might wanna look into Siamese Networks , which is exactly what they do: https://www.cs.cmu.edu/~rsalakhu/papers/oneshot1.pdf

The main idea here is that there are 2 Neural Networks that are trained together, Then, after the training is done. You use both neural networks to extract features of the same images seperately and compare the results to find the similarity.