Phi-4 Mini
Microsoft · February 2025
● activeOpen Sourcedecoder onlytext
Parameters3.8B
Context Window128K tokens
Description
Ultra-compact 3.8 billion parameter model from the Phi-4 family, optimized for running directly on phones, laptops, and edge devices. Despite its tiny size, it performs remarkably well on reasoning and math benchmarks thanks to Microsoft's approach of training on carefully curated, high-quality data rather than simply using more data.
Key Innovations
Distillation
DistillationTraining a smaller 'student' model to mimic a larger 'teacher' model, preserving capability at lower cost.
Reasoning
ReasoningStructured step-by-step problem solving, often using chain-of-thought or tree-of-thought approaches.