Sebastian Raschka’s new project is pushing AI forward by teaching language models to think out loud with clear, step-by-step answers. This method, called structured chain-of-thought reasoning, helps AIs solve math, puzzles, and code problems much better – up to 32% more accurate! Raschka shows how to train these smart models with simple code and thoughtful tips, so even with less computer power, anyone can build and use them. All the tools and guides are free to try, and more lessons are coming soon for anyone who wants to learn.
What is structured chain-of-thought (CoT) reasoning in 2025 LLMs?
Structured chain-of-thought (CoT) reasoning in 2025 LLMs means generating explicit intermediate steps before giving a final answer. This approach boosts performance in tasks like math, symbolic puzzles, and coding, with accuracy gains of 24–32% compared to direct answers.
What counts as “reasoning” in 2025 LLMs?
Raschka defines reasoning as structured chain-of-thought (CoT) generation – the ability to emit explicit intermediate steps before producing a final answer. Recent benchmarks show that CoT-equipped models:
| Task category | Average gain vs. direct answer | Example dataset |
|---|---|---|
| Multi-step math | +32 % accuracy | GSM8K |
| Symbolic puzzles | +28 % | ARC-AGI |
| Code debugging | +24 % | HumanEval-CoT |
Source data from McKinsey’s July 2025 workplace report.
From zero to reasoning model – the learning path
The early-release chapters follow a three-stage sequence:
- Base model primer – start with a 7 B-parameter decoder that already speaks English but has no deliberate reasoning skill.
- Inference-time scaling – add self-consistency sampling and majority voting to squeeze more performance without retraining.
- Lightweight post-training – apply reinforcement learning from human feedback targeted specifically at step-by-step formats, keeping GPU hours below 200 on a single A100.
All code is MIT-licensed and already available on GitHub.
Techniques covered (and why they matter this year)
| Method | Hardware budget | Typical use case | Industry adoption index* |
|---|---|---|---|
| Self-consistency | 3-5x inference tokens | Customer support bots | 68 % |
| Distillation into 1 B “tiny reasoners” | 25 % of original | On-device assistants | 44 % |
| Tool-integrated CoT (Python + LLM) | ~0 extra training | Finance & coding co-pilots | 52 % |
- Adoption index: % of surveyed AI teams piloting the technique in 2025 (source: Ahead of AI Magazine survey, 1 024 respondents, April 2025).
Hands-on demo: a 10-line snippet to add CoT
“`python
excerpt from chapter 2
prompt = “””
Q: Roger has 3 tennis balls. He buys 2 more cans, each with 4 balls. Total?
A: Let’s break this down step by step.
1. Roger already has 3 balls.
2. Each new can contains 4 balls, so 2 cans = 8 balls.
3. Total = 3 + 8 = 11.
So, the answer is 11.
“””
“`
Running this prompt through the scaffold code increases correct answers from 62 % to 87 % on a 100-question grade-school math set.
Roadmap and next drops
Raschka’s Manning page lists upcoming chapters on:
- reinforcement learning with verifiable rewards (autumn 2025)
- scaling laws for reasoning (winter 2025)
- production deployment patterns (early 2026)
Early readers gain free updates; final print edition slated for late 2026.
Sebastian Raschka’s latest book, “Build a Reasoning Model (From Scratch),” is already reshaping how practitioners approach next-generation LLM reasoning. Here are the five most pressing questions the early chapters answer, along with exclusive insights from the ongoing 2025 release.
What exactly is “reasoning” in an LLM, and why does it matter now?
According to the live text, reasoning here means generating explicit intermediate steps (so-called chain-of-thought) before the final answer. This turns pattern-matching models into step-by-step problem solvers, crucial for:
- Multi-step arithmetic
- Logic puzzles
- Advanced code generation tasks
The first live chapter stresses that as of 2025, inference-time reasoning is topping benchmark charts, making this skill non-negotiable for production-grade applications.
How does the book teach reasoning augmentation without starting from zero?
Instead of training a costly new model, the book takes any pre-trained base LLM and adds reasoning capabilities layer by layer. Early readers report that:
- Live code snippets (Python + PyTorch) let you start experimenting within minutes
- Chapter 1 notebook already shows how to inject Think step-by-step prompts and measure accuracy gains
- Official GitHub repo contains the exact
reasoning.pyfile being updated weekly
Which practical techniques are demonstrated first?
The available material spotlights three 2025-proven methods:
- Inference-time scaling – running multiple reasoning passes and scoring them
- Reinforcement learning (RL) fine-tuning – rewarding correct reasoning chains
- Knowledge distillation – compressing bigger reasoning models into smaller ones
These align with the newest industry white papers that cite up to 32 % accuracy jumps on GSM-8k math tasks.
Is the content only for researchers?
No. Early feedback from Manning’s MEAP program shows the book targets practitioners and data scientists:
- 400 LOC starter template turns theory into an executable prototype
- Hands-on exercises guide you from zero to a working reasoning model in under 3 hours
- Slack channel (linked inside chapter 2) already has 1,200+ early readers sharing tweaks
Where can I access the first chapters right now?
As of today:
- First three chapters are live on the Manning Early Access page
- Companion GitHub repo is at github.com/rasbt/reasoning-from-scratch with weekly tags
- Author’s announcement thread offers direct links and changelog
With reasoning becoming the fastest-growing specialization in LLM engineering (McKinsey, July 2025), Raschka’s project arrives exactly when practitioners need a practical, code-first guide.
