IBM details how Granite 4.2 was built: dense 3B, 8B and 30B models with agentic RL and thinking modes
An unusually open account of a model recipe, from token counts to reinforcement learning stages, shows what goes into a small agent-capable LLM.
IBM has published a detailed account of how its Granite 4.2 language models were built, in a post on the Hugging Face blog dated 25 August 2026. The release comprises three dense, decoder-only models at 3, 8 and 30 billion parameters, licensed under Apache 2.0 and available on Hugging Face and GitHub, with quantised variants in FP8, FP4 and GGUF formats.
The models use a standard transformer design with grouped query attention using 40 attention heads and 8 key-value heads, rotary position embeddings with a theta of 10,000,000, SwiGLU activations, RMSNorm and separate input and output embeddings, trained in bfloat16. The context window is 131,072 tokens, and the post says context was extended to 512K during base model pre-training.
Pre-training covered roughly 15 trillion tokens on top of the Granite 4.1 base models, using a five-phase strategy. Supervised fine-tuning used about 7.2 million samples, around 100 billion tokens, of which 31.6 percent were agentic and 68.4 percent non-agentic. Post-training then applied multi-stage reinforcement learning using asynchronous GRPO: a foundational stage covering maths, code, STEM, tool calling and reasoning; skill boosters for instruction following and competitive coding; an agentic stage for the 8B and 30B models only, run in real environments including software engineering agents, terminal operations and web search; and a final RLHF alignment stage for preference and safety.
New in 4.2 are explicit chain-of-thought reasoning with thinking and non-thinking modes, a low-effort thinking option, native OpenAI-format tool calling and multi-turn agent interaction in sandboxed environments. IBM reports scores including 86.67 percent on AIME25 and 47.67 percent on SWE-Bench Verified for the 8B model, and 89.17 percent and 57.00 percent respectively for the 30B. Twelve languages are supported.
Why it matters
The post is notable less for the benchmark numbers than for how openly it documents the recipe, down to the split between agentic and non-agentic fine-tuning data. Agentic reinforcement learning in real environments, once confined to frontier labs, is now part of an Apache-licensed model family at 8 billion parameters. The addition of adjustable thinking effort shows vendors treating inference cost as a first-class design dimension. Small open models that can call tools reliably are what make locally hosted agents practical.
The post is notable less for the benchmark numbers than for how openly it documents the recipe, down to the split between agentic and non-agentic fine-tuning data.
Pre-training
About 15 trillion tokens on top of Granite 4.1 base models, in five phases, with context extended to 512K
Supervised fine-tuning
About 7.2 million samples (~100B tokens); 31.6% agentic and 68.4% non-agentic data
Foundational RL
Asynchronous GRPO on maths, code, STEM, tool calling and reasoning
Skill boosters and agentic RL
Instruction following and competitive coding; agentic RL in real environments for 8B and 30B only
RLHF alignment
Final preference and safety tuning before release under Apache 2.0
What you can learn from this
- Dense versus mixture-of-experts. A dense model activates every parameter for every token, while a mixture-of-experts model routes each token to a subset of specialised sub-networks. Dense designs are simpler to serve and quantise, and at 3 to 30 billion parameters they fit on a single accelerator or even a laptop, which suits the local and edge deployments IBM targets. The trade-off is that a dense model spends compute uniformly, whereas MoE can scale total capacity without proportional cost per token. Knowing which you are running tells you what memory and latency to expect.
- Grouped query attention and the KV cache. During generation, a transformer stores keys and values for every previous token so it does not recompute them, and that cache grows with context length and with the number of key-value heads. Grouped query attention lets several query heads share one key-value head, so 40 query heads backed by 8 key-value heads cuts cache memory by five times compared with full multi-head attention. That is what makes a 131,072-token context feasible on modest hardware. The cost is a small loss in expressiveness that training usually compensates for.
- Why RoPE theta is so large. Rotary position embeddings encode position by rotating query and key vectors, and the base frequency, theta, sets how slowly the rotation cycles across positions. A larger theta spreads the encoding over longer distances, allowing the model to distinguish positions hundreds of thousands of tokens apart. Raising theta and continuing pre-training on long sequences is the standard way to extend context. This is why the post mentions a 512K extension during pre-training even though the shipped window is 131K.
- The post-training pipeline. Supervised fine-tuning teaches format and behaviour from curated examples; reinforcement learning then optimises for outcomes that can be checked, such as a correct answer, passing tests or a successful tool call. GRPO compares groups of sampled responses to the same prompt and pushes the model toward the better ones, and running it asynchronously keeps accelerators busy while environments execute. Agentic RL extends this to multi-step tasks in real sandboxes, where the reward arrives only after many actions. Each stage builds on the last, which is why the order matters.
- Thinking modes trade tokens for accuracy. Letting a model write out its reasoning before answering improves results on maths and code but multiplies output tokens and latency. Offering a non-thinking mode and a low-effort mode lets the caller decide per request how much compute to spend. This mirrors the broader move toward treating test-time compute as a dial rather than a fixed property of a model.
We teach this
How to use this in practice
- Run a quantised Granite 4.2 locally and compare modes. Pull the GGUF variant of the 8B model into llama.cpp or Ollama, then send the same five prompts, including two maths problems and one coding task, with thinking enabled and disabled. Log output tokens and wall-clock time for each. Repeat the coding task in low-effort mode to see whether it lands between the other two. Done looks like a table showing accuracy, token count and latency per mode, so you can see what reasoning costs.
- Build a minimal tool-calling loop. Write a script of under 60 lines that defines one function, such as a unit converter, in OpenAI-format tool schema, sends a user request, executes the tool call the model returns, and feeds the result back for a final answer. Then add a second tool and check that the model picks the right one for each of three requests. Done looks like a transcript where the model calls the tool with correct arguments and uses the result in its reply.
- Measure KV cache growth. Load the model with a small context limit, then increase it in steps of 8K up to the maximum your hardware allows, recording memory use at each step. Plot memory against context length. Compare the slope with what you would predict from the model's layer count and its 8 key-value heads. Done looks like a roughly linear chart that lets you predict the memory needed for a given prompt size.
- Reproduce a benchmark slice. Take 20 questions from a public evaluation set such as MMLU-Pro, run them through the local model, and score them by hand or with a script. Compare your result with the reported figure and note the reasons for any gap, such as quantisation or prompt format. Run it a second time with a different sampling temperature to see how much the score moves. Done looks like a short note with your score, the published score and two hypotheses for the difference.
Sources
- Granite 4.2 LLMs: How They're Built — Hugging Face
Our reporting is an original summary; full coverage is at the links above.
Don't just read about it — build it.
Square 1 teaches the skills behind the headlines, with every line of your work graded by AI. Find your starting point in 3 minutes.
Get your free skill report