AWS describes near-random wiring and intent-driven routing behind its newest data centre networks
Multipath routing, deliberate randomness and flat-rate pricing show how a hyperscaler balances resilience and cost.
Amazon Web Services has given a rare look at how it builds and operates the networks inside its data centres, in an article published by The Register on Saturday 29 August. Corey Quinn visited an AWS networking lab and spoke with Matt Rehder, the company's vice president of global network engineering, and Stephen Callaghan, a senior principal network development engineer.
The design AWS described is a flat, single-tier network that is wired in a deliberately near-random pattern. According to the article, this approach improves resilience and lowers cost, and AWS says it delivers up to a 40 percent energy efficiency improvement over traditional designs. Racks arrive as pre-assembled units, which removes manual hardware installation on site. The design is now the default for most new AWS data centre builds.
Two pieces of software make the topology usable. A control plane called SIDR, for Scalable Intent-Driven Routing and pronounced "cider", decides how traffic should move, and a forwarding system called Spraypoint selects among paths. Between any two points there can be thousands of equivalent routes, and the forwarding layer spreads traffic across them.
The interview also covered pricing. AWS said it has not raised prices on existing EC2 SKUs since 2020, and that the Graviton5 generation carried a 9 percent increase over the previous generation. The company said it kept the margin improvement from cheaper networking rather than passing it through. For its AWS Interconnect multicloud product, AWS is charging a flat hourly rate rather than per gigabyte, with a free tier of one 500 Mbps port per provider. AWS said customers dislike rate-based network charges because they are hard to predict, which is why new network products are moving to flat-rate pricing.
Why it matters
Hyperscale networking is diverging from the tiered designs still common in enterprise data centres. Randomised topology plus intelligent routing trades neat diagrams for statistical resilience, which is only possible when software controls the fabric end to end. The pricing comments show that predictability is becoming a competitive feature in cloud networking. Anyone building on the cloud benefits from understanding both the topology and the bill it produces.
Hyperscale networking is diverging from the tiered designs still common in enterprise data centres.
Intent: SIDR control plane
Scalable Intent-Driven Routing computes how traffic should move across the fabric.
Forwarding: Spraypoint
Selects among thousands of equivalent routes between any two points and spreads traffic across them.
Fabric: flat, single-tier, near-random wiring
Deliberately randomised links for resilience and cost; AWS cites up to 40% better energy efficiency.
Hardware: pre-assembled rack units
Racks arrive built and cabled, removing manual installation on site; now the default for most new builds.
What you can learn from this
- Equal-cost multipath spreads load and absorbs failures. In a traditional three-tier network, traffic between two servers follows a small number of paths through aggregation and core switches, so a failure on one path removes a large share of capacity. When a fabric offers thousands of equal-cost routes, a forwarding engine can hash each flow onto a path and the loss of any single link removes only a tiny fraction of capacity. This behaves well because failure impact is proportional to one path's share of the total. The trade-off is that the control plane has to know about all of those paths and keep them consistent.
- Why randomness helps. A regular, symmetric topology has structured worst cases: certain traffic patterns saturate the same links, and certain failures hit the same neighbourhoods. A near-random wiring pattern has no such structure, so hotspots and correlated failures are less likely. This is the same intuition behind expander graphs in mathematics, where random connections give short paths between any two nodes. It only works when the routing software can make sense of the resulting topology, which is why the wiring and the control plane are designed together.
- Intent-driven control planes separate what from how. In an intent-driven system, operators declare an outcome such as "these two endpoints must reach each other at this capacity" and the controller computes the specific routes. The forwarding hardware then applies those decisions at line rate. Splitting the roles lets the control logic evolve independently of switch silicon and lets the fabric be reprogrammed without rewiring. It also means the controller is a critical dependency, so it has to be built for high availability.
- Pre-assembled infrastructure moves work off the critical path. Assembling racks with cabling and switches in a factory and delivering them as units means fewer field errors, faster deployment and consistent quality. It works because a factory can test a configuration before it ships and repeat it exactly, while on-site technicians work under time pressure with less tooling. The same idea underlies modular data halls and prefabricated power skids.
- Metered versus flat-rate network pricing. Per-gigabyte charges track usage closely but make bills hard to forecast, especially for bursty workloads. A flat hourly rate for a port of known capacity converts a variable cost into a fixed one, which is easier to budget but can cost more when utilisation is low. Providers choose based on what they can predict about their own costs and what customers will accept. Reading the pricing model tells you which risk each side is carrying.
We teach this
How to use this in practice
- Trace the paths in your own network. From a laptop, run
traceroute(ortracerton Windows) to a service you use several times over a few minutes and note whether the intermediate hops change. Then runmtrif it is available to see loss and latency per hop. Repeat the test at a different time of day, since path selection can change with load. Done looks like a note recording how many distinct paths you observed and where the variation occurs. - Draw two topologies and count failure impact. Sketch a classic three-tier network with two core switches and a flat fabric with eight equivalent paths between the same two racks. For each, remove one link and calculate what fraction of capacity is lost. Then remove two links and note whether the tiered design loses connectivity entirely while the fabric only loses capacity. Done looks like two diagrams with a percentage written next to each.
- Audit one cloud network bill. Open the billing console for any cloud account you have, filter to data transfer and networking line items, and classify each as metered (per GB) or flat (per hour or per port). Include any charges for cross-zone or cross-region traffic, which are often the largest metered items, and estimate what the same month would cost under the other model. Note which items you could not classify without reading the provider's pricing page. Done looks like a short table with the two totals side by side.
- Try equal-cost routing in a lab. Using a network simulator such as Containerlab or GNS3, or plain Linux with
ip route add 10.0.0.0/24 nexthop via A nexthop via B, configure two parallel paths between hosts and confirm withip route showand repeated pings that traffic uses both. Then take one path down and check that connectivity survives. Done looks like counters incrementing on both paths and a ping that keeps answering when one link is removed.
Sources
Our reporting is an original summary; full coverage is at the links above.
Don't just read about it — build it.
Square 1 teaches the skills behind the headlines, with every line of your work graded by AI. Find your starting point in 3 minutes.
Get your free skill report