d-Matrix to link Raptor inference chips into Nvidia racks through NVLink Fusion
A specialist inference chip joining Nvidia's interconnect shows that rack fabrics, not just silicon, now decide who wins AI hardware deployments.
AI inference chip designer d-Matrix will connect its next-generation Raptor accelerators to Nvidia's rack infrastructure using the NVLink Fusion interconnect, the two companies announced, according to a ServeTheHome report published on 12 September 2026.
ServeTheHome reported that Raptor is a generative AI inference processor built around d-Matrix's 3D-DRAM memory technology, with low-latency fabric links inside a card and between cards. The chip was shown at Hot Chips 2026. Under the arrangement, Raptor systems would sit inside Nvidia MGX racks alongside Nvidia's Vera CPUs, ConnectX-9 network adapters, BlueField-4 data processing units, NVLink switch trays and Spectrum-X Ethernet switching. NVLink Fusion is Nvidia's programme for letting other companies' chips scale up and out within those racks without designing their own networking. No bandwidth figures, memory capacities or delivery dates were given.
The announcement arrived in the same week that The Register reported on a United States Department of Justice antitrust probe into Nvidia's roughly $20 billion arrangement with inference chip company Groq, first reported by the New York Times on 9 September. That deal, announced in late 2025, combined a technology licence with the hiring of key Groq engineers while leaving Groq's inference service intact. The Register noted that Nvidia contributed its MGX rack designs to the Open Compute Project in late 2024 and opened NVLink Fusion licensing in mid-2025, and listed Cerebras, SambaNova and d-Matrix among the alternative inference accelerator makers now pairing with larger partners.
Why it matters
Specialist inference silicon is increasingly competing on how it plugs into an existing rack rather than on raw chip performance alone. NVLink Fusion lets Nvidia benefit from rivals' accelerators because the CPU, networking and rack design in such deployments remain its own. For buyers, that widens the choice of accelerators while narrowing the choice of fabric. The regulatory attention on Groq shows that the structure of these partnerships is now under as much scrutiny as the hardware.
Specialist inference silicon is increasingly competing on how it plugs into an existing rack rather than on raw chip performance alone.
Scale-out network
Spectrum-X Ethernet switching with ConnectX-9 adapters and BlueField-4 DPUs joining racks across the hall.
Scale-up fabric
NVLink Fusion links and NVLink switch trays joining accelerators inside the rack.
Compute nodes
Nvidia Vera CPUs paired with d-Matrix Raptor inference accelerators.
Accelerator memory
d-Matrix 3D-DRAM keeping model weights close to the compute for the decode phase.
What you can learn from this
- Scale-up and scale-out are two different interconnect problems. Scale-up joins a handful of accelerators into what behaves like one very large device, with memory-like latency and very high bandwidth, which is the job of NVLink and the switch trays in a rack. Scale-out joins many such groups across a data hall over Ethernet or InfiniBand, where bandwidth per link is lower and software has to tolerate the latency. A model that fits one scale-up domain runs very differently from one that must be sharded across the network.
- Inference is usually limited by memory bandwidth, not arithmetic. Generating each token in a large language model requires reading most of the model's weights from memory once, while doing a relatively small amount of maths per weight. That makes the decode phase memory-bound: the chip spends its time waiting for bytes rather than multiplying them. This is why inference-focused designs such as Groq's SRAM-heavy processors and d-Matrix's stacked DRAM concentrate on moving weights closer to the compute.
- Data processing units offload the work that is not the model. A DPU is a network card with its own processor cores that handles packet processing, encryption, storage access and tenant isolation. Moving that work off the host CPU leaves the CPU and accelerators free for the workload, and gives the operator a control point that is separate from the machine it manages. In a dense AI rack, the DPU is also where much of the security boundary lives. Cloud providers use the same idea to run their control plane on the card rather than on the tenant's host.
- Open rack specifications and licensed fabrics create platform gravity. Contributing a rack design to the Open Compute Project makes it easy for any vendor to build compatible hardware, which grows the number of things that fit the rack. Licensing the interconnect then means that even a competitor's accelerator arrives with the platform owner's switches, CPUs and management stack around it. The pattern is familiar from earlier eras of computing, where the bus or the socket, not the fastest chip, defined the ecosystem.
- Licence-and-hire deals are a distinct acquisition structure. Rather than buying a company outright, an acquirer licenses its technology and hires its key engineers, leaving a legal entity and some business behind. The structure can move faster than a full merger and may avoid some review thresholds, which is exactly why regulators have begun examining it. Understanding the structure helps you read the news about who really controls a technology.
We teach this
How to use this in practice
- Map the interconnect topology of a machine you have access to. On a Linux system with Nvidia GPUs, run
nvidia-smi topo -mand read the matrix showing whether each pair of devices is joined by NVLink, PCIe within one host bridge, or a path that crosses CPU sockets. On a laptop,lspci -tvshows the same idea for the PCIe tree. Note which links would have to change if you added a second accelerator. Done looks like a hand-drawn diagram of the boxes and links that matches the tool's output. - Measure memory-bound behaviour with a local model. Install llama.cpp or a similar runtime, load a small quantised model, and record tokens per second while generating a single stream. Then run four to eight concurrent requests and record the total throughput. Done looks like a two-row table showing that aggregate throughput rises with batch size while per-stream speed falls, which is the signature of a memory-bound workload.
- Calculate the arithmetic intensity of one forward pass. For a model you use, take the parameter count, multiply by the bytes per weight at your quantisation to get bytes read per token, and multiply the parameter count by two to estimate floating point operations per token. Divide operations by bytes to get intensity, then compare it with your hardware's peak compute divided by memory bandwidth. Done looks like a single number that tells you whether that chip will be waiting on memory or on maths for that model.
- Draw a reference AI rack from the parts named in the announcement. Sketch a rack with CPUs, accelerators, an NVLink switch tray, a DPU per node and a top-of-rack Ethernet switch, and label which links are scale-up and which are scale-out. Add a note on where a third-party accelerator would plug in and what it would still depend on. Done looks like a diagram you could use to explain to a colleague why the fabric, not just the chip, decides the deployment.
Sources
- d-Matrix Joins the NVIDIA NVLink Fusion Platform — ServeTheHome
- Nvidia's Groq acquihire is on the DOJ's radar, but it's already too late — The Register
Our reporting is an original summary; full coverage is at the links above.
Don't just read about it — build it.
Square 1 teaches the skills behind the headlines, with every line of your work graded by AI. Find your starting point in 3 minutes.
Get your free skill report