AI Insights
Trends, patterns, and analysis derived from 236 models, 53 papers, and 29 hardware milestones.
The Cambrian Explosion
With 236 models now tracked, the AIAIArtificial Intelligence — the broad field of computer science focused on building systems capable of performing tasks that typically require human intelligence. landscape went from a handful of research projects to an industry producing 80 new models per year. 2023 was the inflection point — the year ChatGPT's success triggered an industry-wide arms race. Every major tech company, from Apple to Amazon, scrambled to release their own models. 67 organizations across 10+ countries are now competing.
The Open Source Revolution
In 2021, every major model was closed and proprietary. By 2024, open-weightopen-weightModel weights are publicly released but training data/code may not be. Enables fine-tuning but not full reproduction. and open-source models made up the majority of releases. Meta's LLaMA leak in 2023 was the spark — once researchers could study and fine-tune frontierfrontierThe absolute leading edge of AI model capability, representing state-of-the-art parameters, compute, and benchmark performance.-class models, the community produced an explosion of derivatives. This democratization may be the most consequential trend in AIAIArtificial Intelligence — the broad field of computer science focused on building systems capable of performing tasks that typically require human intelligence. history.
The Deepest Family Trees
OpenAI's GPT family is the deepest evolutionary tree in AIAIArtificial Intelligence — the broad field of computer science focused on building systems capable of performing tasks that typically require human intelligence., with 14 generations from GPT-1 to GPT-5.5. This isn'tTTemperature — a hyperparameter dividing the logits to scale next-token selection randomness. just version numbering — each generation represents genuine architectural or training breakthroughs. Anthropic's Claude lineage, while shorter at 10 generations, shows the fastest iteration pace.
The MoE Takeover
Mixture-of-Expertsmixture-of-expertsArchitecture where only a fraction of the model's parameters are active for each input, allowing massive scale with lower compute. started as a 2017 paper and was mostly ignored. Mixtral 8x7B's viral success in late 2023 proved MoEMoEMixture of Experts — architecture where only a fraction of parameters activate per input, enabling massive scale at lower compute cost. could deliver GPT-4-class quality at a fraction of the inferenceInferenceUsing a trained model to generate predictions or outputs (as opposed to training it). cost. Within 12 months, MoEMoEMixture of Experts — architecture where only a fraction of parameters activate per input, enabling massive scale at lower compute cost. became the default architecture for any model over 100B parameters — adopted by DeepSeek V2/V3, Grok, DBRX, Arctic, and Qwen.
The Context Window Explosion
In 2018, GPT-1 had a 512-tokentokenA basic unit of text (such as a word or subword fragment) processed by a language model. context windowContext windowThe maximum number of tokens a model can process in a single input. Ranges from 2K to 10M+.. By 2025, Gemini offered 10 million tokens — enough to process entire codebases or dozens of novels at once. This wasn'tTTemperature — a hyperparameter dividing the logits to scale next-token selection randomness. just incremental improvement; it required fundamental innovations like Flash AttentionFlash AttentionAn IO-aware exact attention algorithm that's 2-4× faster by minimizing GPU memory reads/writes. and rotary position embeddings. The practical impact: AIAIArtificial Intelligence — the broad field of computer science focused on building systems capable of performing tasks that typically require human intelligence. went from answering single questions to analyzing entire projects.
67 Companies, One Race
What started as a two-horse race between OpenAI and Google has become a global competition spanning 67 organizations. 21 models come from Chinese labs — DeepSeek alone has 7 entries, while Alibaba's Qwen and Zhipu AIAIArtificial Intelligence — the broad field of computer science focused on building systems capable of performing tasks that typically require human intelligence.'s GLM families are rapidly expanding. Community contributors and startups (Nous Research, Eric Hartford) punch far above their weight through fine-tuningFine-tuningAdapting a pre-trained model to a specific task or domain by training on additional data. and abliterationabliterationRemoving safety guardrails from a model through targeted fine-tuning or weight manipulation. Controversial but popular in open-source community. techniques.
Innovation Pipeline: Paper to Product
The pipeline from research paper to production model has dramatically accelerated. Early innovations like RLHFrlhfReinforcement Learning from Human Feedback — training models to align with human preferences by having humans rank outputs. took 5+ years to go mainstream. Now, architectures like Mamba go from paper to production in months. This compression is both exciting (faster progress) and concerning (less time for safety evaluation).
The Modality Matrix
Early AIAIArtificial Intelligence — the broad field of computer science focused on building systems capable of performing tasks that typically require human intelligence. was text-only. Now, over half of new models handle multiple modalities — images, audio, video, and code. The trend is unmistakable: the future of AIAIArtificial Intelligence — the broad field of computer science focused on building systems capable of performing tasks that typically require human intelligence. is models that can see, hear, speak, code, and reason simultaneously. The arrival of models like GPT-4o (text+image+audio) and Gemini 2.0 (text+image+video) marks the beginning of truly general-purpose AIAIArtificial Intelligence — the broad field of computer science focused on building systems capable of performing tasks that typically require human intelligence..
The Reasoning Revolution
The China Factor
The Efficiency Revolution
The AIAIArtificial Intelligence — the broad field of computer science focused on building systems capable of performing tasks that typically require human intelligence. industry hit a wall: training ever-larger models became prohibitively expensive. The responseresponseThe output text generated by a language model in reply to a prompt. was an efficiency revolution. DistilBERT showed you could compress BERT to 60% of its size while keeping 97% of its capability. ALBERT proved parameterparameterA variable or weight inside a neural network that is adjusted during training to store knowledge. sharing could slash model size 18×. Mixture-of-Expertsmixture-of-expertsArchitecture where only a fraction of the model's parameters are active for each input, allowing massive scale with lower compute. architectures (Mixtral, DeepSeek-V2) activate only a fraction of parameters per queryqueryIn self-attention, the vector representing the current token seeking context from other parts of the sequence.. LoRALoRALow-Rank Adaptation — an efficient fine-tuning technique that adds small trainable matrices to frozen model weights. made fine-tuningFine-tuningAdapting a pre-trained model to a specific task or domain by training on additional data. accessible on consumer GPUs. The Chinchilla paper proved most models were undertrained relative to their size. The new mantra: smaller, smarter, cheaper.
The Safety Imperative
From Text to Everything
AIAIArtificial Intelligence — the broad field of computer science focused on building systems capable of performing tasks that typically require human intelligence. has fragmented from one thing (text prediction) into a dozen specialized disciplines. EmbeddingembeddingA high-dimensional vector representing the semantic meaning of a token, grouping similar words close together in vector space. models (text-embeddingembeddingA high-dimensional vector representing the semantic meaning of a token, grouping similar words close together in vector space.-3, BGE) power every search engine and RAGRAGRetrieval-Augmented Generation — combining a language model with a search/retrieval system to ground responses in external knowledge. pipeline. Safety models (Llama Guard, ShieldGemma) act as AIAIArtificial Intelligence — the broad field of computer science focused on building systems capable of performing tasks that typically require human intelligence. immune systems. Robotics models (RT-2, PaLM-E) bridge language and physical action. Music generators (Suno, Udio), speech synthesizers (VALL-E, ElevenLabs), and coding agents (Devin, Cursor, SWE-Agent) each represent billion-dollar verticals. The 'foundation model' era is giving way to an era of specialized, deeply integrated AIAIArtificial Intelligence — the broad field of computer science focused on building systems capable of performing tasks that typically require human intelligence. products.
The Era of Autonomous Agents
Hardware Specialization
What's Next: Emerging Patterns
Based on the trajectories we've tracked, here are the patterns most likely to define AI's next chapter.
38 models already have agenticagenticModels that can autonomously plan, execute multi-step tasks, use tools, and self-correct without human intervention. capabilities — expect this to become the default interaction model.
ParameterparameterA variable or weight inside a neural network that is adjusted during training to store knowledge. growth is plateauing; efficiency (MoEMoEMixture of Experts — architecture where only a fraction of parameters activate per input, enabling massive scale at lower compute cost., Mamba, MLAMLAMulti-head Latent Attention — DeepSeek's innovation that compresses key-value caches into a low-rank latent space.) is the new frontierfrontierThe absolute leading edge of AI model capability, representing state-of-the-art parameters, compute, and benchmark performance.. Smaller, smarter models are winning.
Open-source share peaked at 62% but dropped to 31% in 2026 as frontierfrontierThe absolute leading edge of AI model capability, representing state-of-the-art parameters, compute, and benchmark performance. labs restrict access to their most powerful models.
The 29 hardware milestones show compute doubling every ~18 months, but model demands are growing faster.
Coding tools, search engines, music generators — AIAIArtificial Intelligence — the broad field of computer science focused on building systems capable of performing tasks that typically require human intelligence. is fragmenting into specialized, deeply integrated products that do one thing exceptionally well.
The US-China AIAIArtificial Intelligence — the broad field of computer science focused on building systems capable of performing tasks that typically require human intelligence. race will intensify — Chinese open-weightopen-weightModel weights are publicly released but training data/code may not be. Enables fine-tuning but not full reproduction. models (DeepSeek, Kimi K2) are already matching Western closed models on keykeyIn self-attention, the vector representing what information a token contains, matched against queries to compute attention weights. benchmarks.
Every industry vertical will have its own foundation model — legal AIAIArtificial Intelligence — the broad field of computer science focused on building systems capable of performing tasks that typically require human intelligence., medical AIAIArtificial Intelligence — the broad field of computer science focused on building systems capable of performing tasks that typically require human intelligence., financial AIAIArtificial Intelligence — the broad field of computer science focused on building systems capable of performing tasks that typically require human intelligence. — each trained on domain-specific data at scale.