Kimi K2
Moonshot AI · October 10, 2024
● activeClosedmixture of expertstext
Parameters1T
Context Window2M tokens
Description
Frontier Mixture-of-Experts (MoE) model by Moonshot AI supporting massive context windows with high retrieval accuracy.
Key Innovations
MoE
MoEArchitecture where only a fraction of the model's parameters are active for each input, allowing massive scale with lower compute.
Long Context
Long ContextAbility to process very long inputs (100K+ tokens), enabling analysis of entire codebases or books.