Subscribe
Sign in
Home
Notes
LLM Gallery
Support
LLMs From Scratch Book
Reasoning From Scratch Book
Archive
About
The Big LLM Architecture Comparison
From DeepSeek-V3 to Kimi K2: A Look At Modern LLM Architecture Design
Jul 19, 2025
•
Sebastian Raschka, PhD
1,959
96
173
Most Popular
View all
Understanding Reasoning LLMs
Feb 5, 2025
•
Sebastian Raschka, PhD
1,316
46
123
Understanding and Coding Self-Attention, Multi-Head Attention, Causal-Attention, and Cross-Attention in LLMs
Jan 14, 2024
504
41
20
Components of A Coding Agent
Apr 4
•
Sebastian Raschka, PhD
852
63
87
Latest
Top
Discussions
Recent Developments in LLM Architectures: KV Sharing, mHC, and Compressed Attention
From Gemma 4 to DeepSeek V4, How New Open-Weight LLMs Are Reducing Long-Context Costs
May 16
•
Sebastian Raschka, PhD
298
16
31
My Workflow for Understanding LLM Architectures
A learning-oriented workflow for understanding new open-weight model releases
Apr 18
•
Sebastian Raschka, PhD
66
4
5
Components of A Coding Agent
How coding agents use tools, memory, and repo context to make LLMs work better in practice
Apr 4
•
Sebastian Raschka, PhD
852
63
87
A Visual Guide to Attention Variants in Modern LLMs
From MHA and GQA to MLA, sparse attention, and hybrid architectures
Mar 22
•
Sebastian Raschka, PhD
399
15
34
A Dream of Spring for Open-Weight LLMs: 10 Architectures from Jan-Feb 2026
A Round Up And Comparison of 10 Open-Weight LLM Releases in Spring 2026
Feb 25
•
Sebastian Raschka, PhD
210
13
20
Categories of Inference-Time Scaling for Improved LLM Reasoning
And an Overview of Recent Inference-Scaling Papers
Jan 24
•
Sebastian Raschka, PhD
42
2
The State Of LLMs 2025: Progress, Problems, and Predictions
A 2025 review of large language models, from DeepSeek R1 and RLVR to inference-time scaling, benchmarks, architectures, and predictions for 2026.
Dec 30, 2025
•
Sebastian Raschka, PhD
522
39
55
See all
Ahead of AI
Ahead of AI focuses on machine learning and AI research and is read by more than 150,000 researchers and practitioners who want to stay ahead in a rapidly evolving field.
Subscribe
Ahead of AI
Subscribe
About
Archive
Recommendations
Sitemap
This site requires JavaScript to run correctly. Please
turn on JavaScript
or unblock scripts
