Nvidia releases Nemotron 3.5 Lightning open-weights model and NeMo Switchyard router
Routing each step of a workflow between a cheap fast model and a frontier one is becoming an architecture pattern rather than a hack.
Nvidia has released Nemotron 3.5 Lightning, an open-weights language model, alongside NeMo Switchyard, a router that decides which model should handle each step of an agent's workflow, The Next Platform reported on 11 August 2026. The two are positioned as parts of what Nvidia calls a system of models, on the argument that efficient agents should combine several models rather than rely on a single one.
Lightning uses a hybrid architecture that mixes Mamba state-space layers with transformer layers and adds multi-token prediction. Nvidia says it runs on Jetson devices, GeForce RTX graphics cards, DGX Spark and DGX Station systems. The company claims four times the output speed and 30 percent faster task completion compared with comparable models, and up to 30 percent faster than Qwen models. On PinchBench, a benchmark of agent tasks covering coding, research and file management, Nvidia reports the model outperforming Google DeepMind's Gemma and other frontier models. The model can be post-trained on custom data through the NeMo platform, and Nvidia has published its training data and techniques along with a dataset called Nemotron-RL-Agentic-Terminal-Pivot for training coding agents.
Lightning joins a Nemotron 3 family that includes Nano, a small model released in December 2025, Super, a 120-billion-parameter mixture-of-experts model, and Ultra, a 550-billion-parameter reasoning model.
NeMo Switchyard routes each step of a workflow to a model based on quality, latency and cost. Nvidia claims the router delivers better accuracy at one-third the cost of using Anthropic's Opus 4.8 on its own, and that combining Opus 4.8 with Lightning and Gemma 3 improves both accuracy and cost. Kari Briski, Nvidia's vice president of generative AI, said the best model for a task changes as a workflow evolves, which is why routing needs to be continuous.
Why it matters
The interesting shift here is not one more model but the claim that a router in front of several models beats any single model on cost and accuracy together. That reframes the design question for agent builders from which model to buy to how to decide, per step, which model to call. Open weights that run on a workstation or a single graphics card make the cheap tier of that decision something an individual can experiment with. Expect cost per completed task, rather than benchmark score alone, to become the number vendors compete on.
The interesting shift here is not one more model but the claim that a router in front of several models beats any single model on cost and accuracy together.
Agent step arrives
Each step of a workflow, such as a code edit or a file lookup, is a separate request.
Router scores the options
NeMo Switchyard weighs expected quality, latency budget and cost per call for each candidate model.
Easy steps go to a small fast model
Nemotron 3.5 Lightning handles routine work at low cost on local or edge hardware.
Hard steps go to a frontier model
A larger model such as Opus 4.8 is called only when the router expects the small model to fall short.
Result returns to the loop
The agent continues; the router re-evaluates as the workflow changes.
What you can learn from this
- Hybrid Mamba-transformer architectures. Transformers use attention, which compares every token with every other token and so grows expensive as sequences lengthen. State-space layers such as Mamba process a sequence in a single pass with a fixed-size hidden state, giving roughly linear cost and a constant memory footprint per token. A hybrid keeps a minority of attention layers for tasks that need precise recall of earlier tokens and uses state-space layers for the bulk of the work. The result is faster generation and cheaper long contexts at some cost in the exact retrieval that pure attention provides.
- Multi-token prediction. A standard language model predicts one token at a time, and each prediction requires a full pass through the network. Multi-token prediction trains the model to propose several upcoming tokens in one pass, which can then be verified cheaply, so more text emerges per unit of compute. It works because much of ordinary text is predictable enough that a draft of several tokens is often right. The technique explains how a model can claim higher output speed without a smaller parameter count.
- Model routing and cascades. A router examines each request or workflow step and sends it to the model most likely to answer well within a latency and cost budget. The simplest version tries a small model first and escalates to a larger one only when confidence is low, which is called a cascade. This pays off because most requests are easy, so the expensive model is reserved for the hard minority. The difficulty lies in the routing decision itself: a poor router either wastes money on easy tasks or sends hard tasks to a model that fails quietly.
- Open weights versus open source. An open-weights release lets you download the trained parameters and run or fine-tune the model, but it is governed by a licence that may restrict certain uses, and it does not always include the training code or data. Open source in the strict sense would include everything needed to reproduce the model. Publishing training data and techniques, as reported here, moves a release closer to that end of the spectrum. Always read the licence before building a product on any downloaded model.
- Agentic benchmarks. Traditional benchmarks score a single answer to a single prompt, but agents run multi-step loops with tools, files and commands, where early mistakes compound. Benchmarks built around task completion, such as the coding and file-management tasks described here, measure whether the whole loop reached the goal. They behave differently from chat benchmarks because speed, tool-use reliability and recovery from errors all affect the score. A model that is slightly less accurate per step but much faster can finish more tasks within a fixed budget.
We teach this
How to use this in practice
- Build a two-tier router on your own machine. Install a local runner such as Ollama, pull one small model, and write a script that sends each prompt to it first with an instruction to reply with an answer and a confidence between zero and one. If confidence is below a threshold you choose, resend the prompt to a larger model, local or hosted. Log which model answered, the elapsed time and an estimated cost for twenty varied prompts. Done looks like a CSV you can sort to see which prompts escalated and whether the escalations were justified.
- Measure generation speed properly. Using the same local model, time how long it takes to generate 200 tokens for a fixed prompt, repeat five times and take the median, then repeat with a different quantisation level if your runner offers one. Record tokens per second and memory used. Quantisation trades a little accuracy for speed, so also record any change in answer quality on the same prompt. Done looks like a small table that lets you state, with numbers, what one step of a small model costs on your hardware.
- Draw the routed agent. Sketch an agent loop with a planner, three candidate models of different size, a router in front of them and a tool-calling stage after them. Annotate the router with the three signals it uses, expected quality, latency budget and cost per call, and mark where a wrong routing decision would surface and which model a cascade would try first. Done looks like a one-page diagram you could use to explain a cascade to a colleague.
- Read the model card before you run the model. Locate the official model card for Nemotron 3.5 Lightning and write down its licence, context length, supported hardware and the intended and prohibited uses. Compare those with the claims in the article. Done looks like a half-page note with any gap between marketing claims and documented limits highlighted.
Sources
- Nvidia Drives Bang For The Buck With New GenAI Model And Router — The Next Platform
Our reporting is an original summary; full coverage is at the links above.
Don't just read about it — build it.
Square 1 teaches the skills behind the headlines, with every line of your work graded by AI. Find your starting point in 3 minutes.
Get your free skill report