Thank You for investing so much time and effort in this! I personally have learnt so much from your content. I got introduced to your content when my Professor used your LLM from Scratch book as the primary textbook for the new Introductions to LLMs Course in our college. Your open educational resources are really helping many students grow and helping the community take many steps forward worldwide. I know many people who I am sure will hold similar views. Thank You for keeping so much knowledge so very accessible! Love from India
Tip: you need to keep doing some coding, maybe even just hobby at home instead of work where speed is demanded. Why ? So you still know what to look for when judging what the LLM is doing.
100% agree with you here. I think you are just more effective this way. Interestingly, LLMs make learning more accessible because you can ask it questions any time, and it usually gives good answers and assistance for more beginner-level problems. At the same time, the trick is to withstand the temptation to let it do too much of the thinking for you so that your skills will atrophy, which then in turn makes you less productive when using LLMs.
I agree with this perspective. I believe the breakthrough around 2026 will manifest as AI transitioning from a "logical deductive engine" to a "negotiator of meaning." It will move beyond processing facts and inferences to comprehending the value assumptions underpinning different philosophical stances, and become capable of translating, mediating, and synthesizing between these presuppositions.
This means AI would no longer just answer "what is true," but begin to engage with questions like "what is important" and "why it matters." Such deep conceptual alignment between humans and machines would usher in a new paradigm of philosophizing.
This sounds definitely interesting, but it may be a bit ambitious for 2026. But we will see. I think it's going to be another interesting year with perhaps a view more surprises than we have on our bingo cards.
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
A line in the paper reads: “Wait, wait. Wait. That’s an aha moment I can flag here.” — noted in red on page nine as an “epiphany moment” documented in 2025.
On December 26, 2025, this phenomenon was replicated by Google’s Gemini model, which stated: “WAIT! I SAW IT!”
The way you explain and breakdown topics across all your articles and books, is truly something the opensource community and fellow readers is blessed to have.
It's your research, which gives me enough courage to try and understand new LLM architectures, advancements and stay abreast with the industry.
Great post! It does seem like domain specialisation and access to high-quality proprietary data will become a key bottleneck for AI going forward, and one of the biggest constraints for its effective adoption across industries.
Wonderful recap. I was looking for exactly this before it came in (and i read it entirely 😅). Excited for what 2026 holds for the AI community and see you there!
Fantastic breakdown! The obseravtion about RLVR's current limitations with process reward models really resonates with me. We ran into similar issues where PRMs added alot of overhead during training but didnt improve our downstream task performance much. I dunno if this is the case for everyone, but in production settings, that computational cost just doesnt justify the marginal gains right now. The idea of expanding RLVR into domains beyond math/code using secondary LLMs for explanation-scoring is promising tho—it could potentially unlock reasoning in less structured domains where verifiable rewards are harder to define.
Thank you so much for putting this report together. I recently purchased the Build a Reasoning Model (From Scratch) MEAP and plan to work through the first few chapters in January. I truly appreciate the fantastic work you’ve been doing, and I wish you a happy and prosperous New Year.
Ah yes, I meant it would stay fragmented until at least 2026 and then stabilize in 2026-2027. However, it already stabilized more or less this year with everyone adopting MCP.
As a devoted reader of Build A Large Language Model(From Scratch), I am delighted to have progressed from reading the original English version to supporting this "sustainable learning project" by purchasing the physical book!
Likewise, I am eagerly looking forward to the completion of Build A Large Language Model(From Scratch) and hope to gain access to the full preview version soon to embark on this new learning journey!
Thank You for investing so much time and effort in this! I personally have learnt so much from your content. I got introduced to your content when my Professor used your LLM from Scratch book as the primary textbook for the new Introductions to LLMs Course in our college. Your open educational resources are really helping many students grow and helping the community take many steps forward worldwide. I know many people who I am sure will hold similar views. Thank You for keeping so much knowledge so very accessible! Love from India
Thanks for this very kind message!!
Excellent reading for the end of the year! Thank you!
Tip: you need to keep doing some coding, maybe even just hobby at home instead of work where speed is demanded. Why ? So you still know what to look for when judging what the LLM is doing.
100% agree with you here. I think you are just more effective this way. Interestingly, LLMs make learning more accessible because you can ask it questions any time, and it usually gives good answers and assistance for more beginner-level problems. At the same time, the trick is to withstand the temptation to let it do too much of the thinking for you so that your skills will atrophy, which then in turn makes you less productive when using LLMs.
I agree with this perspective. I believe the breakthrough around 2026 will manifest as AI transitioning from a "logical deductive engine" to a "negotiator of meaning." It will move beyond processing facts and inferences to comprehending the value assumptions underpinning different philosophical stances, and become capable of translating, mediating, and synthesizing between these presuppositions.
This means AI would no longer just answer "what is true," but begin to engage with questions like "what is important" and "why it matters." Such deep conceptual alignment between humans and machines would usher in a new paradigm of philosophizing.
This sounds definitely interesting, but it may be a bit ambitious for 2026. But we will see. I think it's going to be another interesting year with perhaps a view more surprises than we have on our bingo cards.
arXiv:2501.12948v1 [cs.CL] 22 Jan 2025
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
A line in the paper reads: “Wait, wait. Wait. That’s an aha moment I can flag here.” — noted in red on page nine as an “epiphany moment” documented in 2025.
On December 26, 2025, this phenomenon was replicated by Google’s Gemini model, which stated: “WAIT! I SAW IT!”
This is truly a remarkable phenomenon!
The way you explain and breakdown topics across all your articles and books, is truly something the opensource community and fellow readers is blessed to have.
It's your research, which gives me enough courage to try and understand new LLM architectures, advancements and stay abreast with the industry.
Thank you for all you do.
Thanks, Shamik!
Great post! It does seem like domain specialisation and access to high-quality proprietary data will become a key bottleneck for AI going forward, and one of the biggest constraints for its effective adoption across industries.
Excellent read for New Year's Eve! Thank you for writing this!
Wonderful recap. I was looking for exactly this before it came in (and i read it entirely 😅). Excited for what 2026 holds for the AI community and see you there!
Thanks!
Fantastic breakdown! The obseravtion about RLVR's current limitations with process reward models really resonates with me. We ran into similar issues where PRMs added alot of overhead during training but didnt improve our downstream task performance much. I dunno if this is the case for everyone, but in production settings, that computational cost just doesnt justify the marginal gains right now. The idea of expanding RLVR into domains beyond math/code using secondary LLMs for explanation-scoring is promising tho—it could potentially unlock reasoning in less structured domains where verifiable rewards are harder to define.
Nice to hear confirmation that PRMs are nice in theory but still have quite some way to go.
Thanks for all the great writing and teaching you do Sebastian. All the best for 2026!
Thank you so much Sebastian!
Thank you so much for putting this report together. I recently purchased the Build a Reasoning Model (From Scratch) MEAP and plan to work through the first few chapters in January. I truly appreciate the fantastic work you’ve been doing, and I wish you a happy and prosperous New Year.
Thanks for getting a copy, and that sounds like a productive start into 2026! Happy New Year!
Great work - thanks!
For this did you mean 2026-2027?
"I expected the ecosystem to remain more fragmented in 2025, until at least 2026."
Ah yes, I meant it would stay fragmented until at least 2026 and then stabilize in 2026-2027. However, it already stabilized more or less this year with everyone adopting MCP.
Great summary and overview. Some of it was outside my grasp but I was able to follow along with most. Appreciate all you do!
By the way, I noticed there wasn't anything on Interpretability here. That might be a nice addition.
Fair. I don't think there was anything ground-breaking this year in this context though.
Lots of "reasoning chain-of-thoughts are not what they seem"-type of articles this year. Things like
Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?, https://arxiv.org/abs/2504.13837
Why Language Models Hallucinate, https://arxiv.org/abs/2509.04664
Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens, https://arxiv.org/abs/2508.01191
Does Math Reasoning Improve General LLM Capabilities? Understanding Transferability of LLM Reasoning, https://arxiv.org/abs/2507.00432
...
But I think this was always obvious, and the articles just formalize what was already known.
But please let me know if I missed some interesting work that I should have mentioned. I am always interested in the interpretability angle of things.
Excellent! Thank you to the author for sharing!
As a devoted reader of Build A Large Language Model(From Scratch), I am delighted to have progressed from reading the original English version to supporting this "sustainable learning project" by purchasing the physical book!
Likewise, I am eagerly looking forward to the completion of Build A Large Language Model(From Scratch) and hope to gain access to the full preview version soon to embark on this new learning journey!