iitb-t5-finetuned-punctuation

This model is a fine-tuned version of google-t5/t5-base on an a english-punctuation restoration dataset dataset. It achieves the following results on the evaluation set:

• Loss: 0.0897
• Bleu: 53.0293

Author: Kaustubh S. Shejole

Model Usage

from transformers import pipeline
# This might accidentally default to a translation task
punctuator_pipeline = pipeline("text2text-generation", model="thenlpresearcher/iitb-t5-finetuned-punctuation")

text = "the morning sky stretched over the city like a quiet sheet of pale blue while people hurried through the streets"
punctuator_pipeline(text,
                   max_length=128)

#output
# [{'generated_text': 'the morning sky stretched over the city like a quiet sheet of pale blue while people hurried through the streets.'}]

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 64
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 3
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu
0.0988	1.0	6441	0.0947	52.8823
0.0879	2.0	12882	0.0910	52.9691
0.0832	3.0	19323	0.0897	53.0293

Framework versions

• Transformers 4.50.0
• Pytorch 2.5.1+cu121
• Datasets 2.21.0
• Tokenizers 0.21.4