Microsoft's Project Zenith Windows build needs 64GB memory and 250GB/s bandwidth for local AI

Unified memory means one pool shared by CPU and GPU. In a conventional PC the processor has its own RAM and a discrete graphics card has separate, smaller video memory, and data must be copied across a bus between them. In a unified design such as Ryzen AI Halo, both engines address the same physical memory, so a large model can be loaded once and used by whichever engine is fastest for the job. That is why a 128GB machine can hold a model that a 24GB graphics card cannot, even if the graphics card is faster at raw arithmetic.

Memory bandwidth, not compute, usually limits local inference speed. Generating each token requires reading essentially every parameter of the model from memory, so the ceiling on tokens per second is roughly bandwidth divided by model size in bytes. A 30-billion-parameter model stored at two bytes per parameter is about 60GB, and at 250GB/s that reads out only a few times per second. This is why Microsoft's requirement lists bandwidth alongside capacity: capacity decides whether the model fits, bandwidth decides whether it is usable.

Quantisation trades precision for fit and speed. Model weights are commonly trained at 16 bits but can be stored at 8 or 4 bits with modest quality loss, halving or quartering both memory needed and bytes read per token. That is the trick that makes a 30B model practical on a 64GB device with room left for the operating system and your tools. It works because most weights carry little information individually, so coarse rounding changes the output surprisingly little.

Local inference changes the cost curve from variable to fixed. Cloud APIs charge per token, so an agent that loops through thousands of tool calls has a bill that scales with activity. A local machine costs the same whether it produces one token or a billion, so the more you use it the cheaper each token becomes. The trade-off is that the local model is usually smaller and slower than the frontier model in the cloud, so the decision depends on the task.

A preinstalled toolchain is a product decision, not a convenience. Bundling WSL 2, Git, Python and Node makes a new device usable within minutes and standardises the environment across a team. It also means those versions are the ones the vendor tests against, which is why such images pin minimum versions such as Python 3.14 and Node 24. Understanding what is pinned tells you what will be supported and what you will have to manage yourself.