Is there an easy way to access the pretrained weights for Google's ViT model?

Question

I'm doing a project where I'm using a ViT. I heard that they need a lot of pretraining to beat out CNNs, so I want to use Google's pretrained weights and load them into a Keras model. However, it doesn't look like theres an easy way to do it and I can't find the model weights on google's github. If I can't get this to work I'll probably try using a Hugging Face model.

Answer 1

So I found this github repo that allows you to load the pre-trained weights of the model. In the documentation there is also a snippet for fine-tuning. Like so:

image_size = 224
model = vit.vit_l32(
    image_size=image_size,
    activation='sigmoid',
    pretrained=True,
    include_top=True,
    pretrained_top=False,
    classes=200
)
# Train this model on your data as desired.