如何在没有预训练 model 的情况下使用 T5 架构（拥抱脸）

Question

I would like to study the effect of pre-trained model, so I want to test t5 model with and without pre-trained weights. Using pre-trained weights is straight forward, but I cannot figure out how to use the architecture of T5 from hugging face without the weights. I am using Hugging face with pytorch but open for different solution.

Answer 1

https://huggingface.co/docs/transformers/model_doc/t5#transformers.T5Model

"Initializing with a config file does not load the weights associated with the model, only the configuration."

for without weights create a T5Model with config file

from transformers import AutoConfig    
from transformers import T5Tokenizer, T5Model    
model_name = "t5-small"    
config = AutoConfig.from_pretrained(model_name)    
tokenizer = T5Tokenizer.from_pretrained(model_name)    
model = T5Model.from_pretrained(model_name)    
model_raw = T5Model(config)