Liquid AI releases LFM2.5-DSpark draft models claiming up to 3.2x faster inference
Speculative decoding shows how a small model guessing ahead can speed up a large one without changing its answers.
Liquid AI has published LFM2.5-DSpark, a set of draft models designed to accelerate inference for its LFM2.5 language models through speculative decoding, in a post on the Hugging Face blog dated 20 August. The release covers three variants: LFM2.5-1.2B-Instruct-DSpark, LFM2.5-2.6B-DSpark and LFM2.5-8B-A1B-DSpark, each with a draft model of roughly 300 million parameters.
Speculative decoding uses a lightweight draft model to propose candidate tokens, which the larger target model then verifies in parallel. According to the post, the DSpark design combines three parts: a DFlash-style parallel backbone, a sequential Markov-chain head that captures dependencies between adjacent tokens, and a confidence-scheduled verifier that stops checking low-confidence guesses when verification would cost more than it saves. The draft models are attention-only networks with five decoder layers plus projection and Markov-chain components, trained for 15 epochs on supervised fine-tuning, chat, code and function-calling data, with training data selected to maximise acceptance rate rather than minimise loss.
The headline figure of 3.18x comes from the 8B-A1B model on the MATH500 dataset running on an NVIDIA H100 80GB GPU with the SGLang framework, where throughput rose from 428 to 1,362 tokens per second. Average gains across datasets on that hardware ranged from 2.54x to 2.67x. For the 2.6B model, the post reports 3.06x on MATH500 (326 to 1,000 tokens per second), 2.56x on HumanEval (326 to 835), and an average 57 percent reduction in function-calling latency. On an Apple M4 Max MacBook Pro using llama.cpp with Metal acceleration, the average speedup was 2.27x.
The models are available on Hugging Face in Safetensors and GGUF formats, with day-one support in llama.cpp and SGLang through open-source implementations.
Why it matters
Inference cost, not training cost, now dominates the economics of deploying language models, and speculative decoding is one of the few techniques that improves speed without changing the model's outputs. Releasing draft models alongside open weights and open inference engines lowers the barrier for small teams to use it. The reported gains on a laptop show the approach is relevant to on-device assistants, not only data centre serving. Expect draft models to become a standard companion artefact for open model releases.
Draft model proposes
A small model quickly guesses several next tokens in a row.
Target model verifies
The full model checks all the guesses in a single parallel forward pass.
Accept the matching prefix
Tokens that agree with what the target would have produced are kept.
Correct and repeat
At the first mismatch the target supplies its own token, then drafting resumes.
What you can learn from this
Token generation is a memory-bound loop. A language model produces text one token at a time, and each step reads every weight in the model from memory to produce a single output. The arithmetic per step is small relative to the bytes moved, so the GPU spends most of its time waiting on memory bandwidth. Speculative decoding exploits this idle capacity by asking the model to check several candidate tokens in one pass, which costs little more than checking one. The weights are read from memory once and applied to several positions, so the expensive part of the step is amortised across the whole draft.
Verification keeps the output distribution unchanged. The target model does not simply trust the draft; it computes what it would have generated at each position and accepts a drafted token only where the two agree under a sampling rule. Where they disagree, the target's own token is used and the rest of the draft is discarded. Because acceptance is decided by the target, the final text is statistically the same as running the large model alone, only faster.
Acceptance rate is the number that matters. The speedup depends on how often the draft's guesses are correct, not on how good the draft is at benchmarks in isolation. This is why the post describes selecting training data to maximise acceptance rather than minimise loss. A draft that mimics the target's habits, including its mistakes, is more useful than one that is independently accurate. There is also a balance to strike, since a longer draft raises the potential gain but increases the chance that an early rejection wastes the rest of the guesses.
Structured text is easier to predict. Mathematical derivations, code and function-call arguments contain long runs of predictable tokens such as brackets, keywords and repeated identifiers. The draft model can guess these with high confidence, so acceptance rates and speedups are highest on tasks like MATH500 and HumanEval. Free-form creative writing, by contrast, has more surprising tokens and benefits less.
Small draft networks can be built from simple parts. The draft described here is attention-only with just five layers, and it borrows from the target's knowledge rather than learning language from scratch. A sequential head that models the dependency between consecutive guesses helps because a wrong early token makes every later guess wrong too. Confidence scheduling then avoids wasting a verification pass when the draft is unsure.
We teach this
How to use this in practice
Run a speculative decoding benchmark on your own machine. Install llama.cpp, download a GGUF target model and its matching draft model, and run the server or command-line tool once with the draft attached and once without, using the same prompt and the same number of output tokens. Record tokens per second from the log for each run, and repeat each run three times to smooth out variance from background processes. Done looks like a small table with two rows showing the baseline and drafted throughput, and the ratio between them, which you can compare against the figures reported in the release.
Measure acceptance rate across prompt types. Using the same setup, run three prompts: a coding task, a short maths derivation and a paragraph of open-ended prose. llama.cpp reports how many drafted tokens were accepted, so divide accepted by proposed for each run. Done looks like three acceptance percentages in your notes and a sentence explaining why the structured prompts scored higher, which is the same pattern the release describes for maths and code benchmarks.
Estimate the memory-bound step time by hand. Take the size of a quantised model file in gigabytes and divide it by your GPU or laptop's memory bandwidth in gigabytes per second. The result approximates the minimum time per token when generating without a draft. Done looks like a predicted tokens-per-second figure that you can compare against what you measured, which makes the memory-bound argument concrete.
Draw the speculative decoding loop. Sketch the four stages from drafting through verification, acceptance and correction, and annotate where the parallel forward pass happens. Add a note explaining why a rejected token discards everything after it, and mark which stage runs on the small model and which on the large one. Done looks like a diagram you could use to explain to a teammate why the technique never changes the quality of the answer while still cutting latency.
Sources
- Up to 3.2x Faster Inference with LFM2.5-DSpark — Hugging Face
Our reporting is an original summary; full coverage is at the links above.
Don't just read about it — build it.
Square 1 teaches the skills behind the headlines, with every line of your work graded by AI. Find your starting point in 3 minutes.
Get your free skill report