DeepSeek V4.1 Flash packs 763 billion parameters while keeping GPU memory needs in check
By parking 196 billion parameters in lookup tables and shrinking the KV cache, DeepSeek shows model size and serving cost are no longer the same thing.
DeepSeek has released V4.1 Flash, and The Register reported on 11 September that the point release brings a redesigned architecture rather than a minor tune-up. The model weighs in at 763 billion parameters, more than two and a half times the size of the V4 Flash it replaces and larger than the V3 and R1 models that put the company on the map in early 2025.
The headline number hides a different story about memory. Of those 763 billion parameters, 196 billion belong to what DeepSeek calls a conditional memory module built from N-grams, groups of sequential tokens that behave a little like learned word associations. Rather than matching the prompt directly, the model performs hash-based lookups into this pool and feeds the resulting vectors into the inference pipeline. Because the tables are queried rather than read in full for every generated token, they can be offloaded to system RAM or storage without a performance hit. The Register's arithmetic puts the theoretical minimum GPU memory at around 567 GB at FP8 precision, down from 763 GB if everything had to stay on the accelerator.
The N-gram weights sit alongside the eight billion parameters that are active when processing a prompt, and they do not raise that active count. Separately, modified attention mechanisms and a causal encoder-decoder design cut key-value cache consumption to between 13 and 25 percent of V4 Flash's requirements, which the report translates as four to eight times as many concurrent users in the same KV cache footprint.
The approach is not unique to DeepSeek. Google uses a related technique, Per-Layer Embedding, in small models, and Alibaba's experimental Qwen 3.8-Flash-Next carries a 51 billion-parameter N-gram pool drawing on a DeepSeek paper from January, an architecture Alibaba says will underpin its Qwen 4 generation.
Why it matters
Parameter count is decoupling from serving cost, and the two now have to be judged separately. Lookup-table memory and leaner caches push the cost of a large model towards commodity RAM rather than scarce GPU memory. If rivals adopt the same recipe, competition shifts to how cleverly a model spends its memory budget, not just how large it is.
Parameter count is decoupling from serving cost, and the two now have to be judged separately.
Active parameters: 8 billion per token
The compute actually performed for each generated token; the N-gram tables do not add to this count.
KV cache: 13 to 25 percent of V4 Flash
Modified attention and a causal encoder-decoder shrink per-user state, allowing four to eight times as many sessions.
Dense weights on GPU: about 567 GB at FP8
The theoretical minimum GPU memory once the lookup tables are offloaded, down from 763 GB.
N-gram lookup tables: 196 billion parameters
Hash-based conditional memory queried sparsely, so it can live in system RAM or storage.
What you can learn from this
- Active parameters, not total parameters, set the compute per token. Mixture-of-experts style designs route each token through a small subset of the network, so a model advertised at hundreds of billions of parameters may only do the arithmetic of a much smaller one per step. The whole model still has to be stored somewhere, which is why memory rather than raw compute often dominates the serving bill. Understanding the split lets you read a model card without being misled by the biggest number.
- The KV cache is the hidden memory cost of long conversations. Transformers keep the keys and values computed for every earlier token so they need not be recomputed when the next one is generated, and that cache grows with context length and with every concurrent user. Attention variants that share or compress those tensors shrink the per-user footprint, which is why a 13 to 25 percent figure translates directly into more simultaneous sessions. Serving capacity is therefore usually bounded by cache, not by the weights.
- Lookup tables trade compute for memory in a way that suits cheap storage. A hashed N-gram table is touched sparsely: each token consults only a handful of entries, so the whole structure never streams through the GPU. Data that is accessed sparsely tolerates slower memory well, which is why it can sit in system RAM or on disk while dense weights stay near the compute. This is the same memory-hierarchy principle that governs CPU caches and database indexes.
- Precision formats such as FP8 halve the bytes per parameter. Each weight stored in eight-bit floating point takes half the space of a 16-bit one, which is how 763 billion parameters map to roughly 763 GB. Lower precision works because inference tolerates small rounding errors once a model is trained, though it is not free and quality checks matter. Knowing the bytes-per-parameter conversion lets you estimate hardware needs from a parameter count in your head.
- Architectural ideas travel between labs quickly when papers are public. The same N-gram memory concept appears in DeepSeek's January paper, in Alibaba's Qwen line and, in a related form, in Google's small models. Open publication means an efficiency gain in one place tends to become an industry default within months. Following the papers, not only the product launches, is how you anticipate what the next generation of models will look like.
We teach this
How to use this in practice
- Do the memory arithmetic for a model you actually use. Take its parameter count from the model card, multiply by bytes per parameter for FP16, FP8 and 4-bit, and write down the three totals. Then add a KV cache estimate for your typical context length using the layer count and hidden size from the config file. Repeat for a model twice the size to see how quickly the numbers outrun a single card. Done looks like a small table showing whether each model fits your GPU at each precision, with the cache row included.
- Run a small quantised model locally and watch the memory. Install a local runner such as llama.cpp or Ollama, pull a small open-weight model in two different quantisations, and observe resident memory in Task Manager or with nvidia-smi while you chat with it. Note how the footprint changes with a long prompt versus a short one, and if the runner exposes a flag for context length or cache quantisation, change it and record the difference. Done looks like a note recording model, quantisation, idle memory and memory after a 2,000-token prompt.
- Draw the inference memory hierarchy. On one page, sketch GPU memory, system RAM and disk as three tiers, then place dense weights, the KV cache and a lookup-table memory in the tier where each belongs and label why. Add an arrow showing what streams through the GPU on every token and what is only touched sparsely. Done looks like a diagram a colleague could read to explain why offloading lookup tables costs so little.
- Read one architecture paper's abstract and method section. Pick a paper on conditional memory or attention-cache reduction, read only the sections that describe what changed, and write a five-line summary in plain words. Done looks like a paragraph you could paste into a team channel that names the mechanism, the memory it saves and one sentence on what it would change for a system you run.
Sources
Our reporting is an original summary; full coverage is at the links above.
Don't just read about it — build it.
Square 1 teaches the skills behind the headlines, with every line of your work graded by AI. Find your starting point in 3 minutes.
Get your free skill report