T5
Google · October 2019
● activeOpen Sourceencoder decodertext
Parameters11B
Context Window512 tokens
Why It Matters
Unified all NLP tasks into a single text-to-text framework, proving that a single model architecture could handle any language task when framed as text generation.
Description
Google's 'Text-to-Text Transfer Transformer' unified all NLP tasks into a single text-to-text format — translation, summarization, classification, and question answering all became 'given this text input, produce this text output.' Available up to 11 billion parameters and became one of the most influential architectures in NLP history.
Key Innovations
Instruction Tuning
Instruction TuningFine-tuning a model on instruction-response pairs so it follows user commands more reliably.
Scaling Laws
Scaling LawsMathematical relationships showing how model performance improves predictably with more data, compute, and parameters.