将 pandas dataframe 转换为火炬数据集

Question

I have a pandas dataframe with the following structure:

path	sentence	speech	input_values	labels
audio1.mp3	This is the first audio	[[0.0, 0.0, 0.0, ..., 0.0, 0.0]]	[[0.00005, ..., 0.0003]]	[23, 4, 6, 11, ..., 12
audio2.mp3	This is the second audio	[[0.0, 0.0, 0.0, ..., 0.0, 0.0]]	[[0.000044, ..., 0.00033]]	[23, 4, 6, 11, ..., 12

The sentence is the transcription of the audio, the speech column is the array representation of the audio, and labels is the number representation of the each letter of the sentence based on a defined vocab list.

I'm fine-tuning a pre-trained ASR model, but when I try to pass the pandas df to the Trainer class and call .train() on it, it errors out (KeyError: 0). From the documentation, it only accepts torch.utils.data.Dataset or torch.utils.data.IterableDataset as train_/eval_dataset arguments. This is how my Trainer definition looks like:

trainer = Trainer(
    model=model,
    data_collator=data_collator,
    args=training_args,
    compute_metrics=compute_metrics,
    train_dataset=ds_train, 
    eval_dataset=ds_test,
    tokenizer=processor.feature_extractor
)

ds_train and ds_test are my training and validation dataframes respectively. I just split my main dataframe (80/20). How can I convert my pandas dataframes into the required Dataset type? I tried tailoring the data_collator class definition to a pandas df but that predictably didn't work either. I'm assuming the train and eval datasets both call the data_collator class when you call .train() on the trainer?

EDIT : I tried using Dataset.from_pandas(ds_train) but it couldn't convert it because I had columns with two-dimensional arrays and it can apparently only convert one-dimensional array values.

Answer 1

Depends on how you will use your labels column. I don't know how your your trainer use these data but I suggest to define your own Dataset class ( https://pytorch.org/tutorials/beginner/basics/data_tutorial.html#creating-a-custom-dataset-for-your-files )

class CustomDataset(Dataset):
    def __init__(self, dataframe):
        self.path = dataframe["path"]
        self.sentence = dataframe["sentence"]
        self.speech = dataframe["speech"]
        self.input_values = dataframe["input_values"]
        self.labels = dataframe["labels"]

    def __len__(self):
        return len(self.text)

    def __getitem__(self, idx):
        path = self.path.iloc[idx]
        sentence = self.sentence.iloc[idx]
        speech = self.speech.iloc[idx]
        input_values = self.input_values .iloc[idx]
        labels = self.labels.iloc[idx]
        return path, sentence, speech, input_values, labels