From-scratch Transformer · Machine Translation
"Śrutam" — that which is heard. A single Transformer, hand-built and trained from zero, that reads eleven Indic scripts and speaks back in English.
Every source sentence carries a language tag — <hi>, <bn>, <ta> — so a single shared encoder learns all eleven at once.
Step 400,000 of 400,000 — checkpoint released as-is, no cherry-picking.
Validation perplexity nearly halved between step 20,000 and step 400,000, once the Noam learning-rate schedule cleared its mid-training plateau.
30 held-out sentences per language, beam search (size 4). Quality tracks training-data volume closely — Bengali had 8.5M pairs, Assamese had 137K.
| Language | Code | BLEU | chrF |
|---|
64K joint SentencePiece BPE spanning all eleven scripts plus English, with language tags reserved as single, unsplittable tokens.
6+6 layers, d_model 768, 12 heads — the original Vaswani et al. architecture, hand-written in PyTorch with no pretrained components.
Source embedding, target embedding, and output projection share one weight matrix — fewer parameters, better cross-lingual generalization.
Beam search, width 4, length-penalty 0.6, run per-language at evaluation time for faithful BLEU/chrF scoring.