OpenAI의 Stargate 인프라 프로젝트는 '단순' 네트워크 아키텍처를 구현하여 백본 라우팅을 간소화하고 GPU 클러스터 성능을 우선한다.

In Mark Handley's view, you should never bet against Ethernet. A member of OpenAI's core networking team and a key architect of the Ethernet-based Multipath Reliable Connection (MRC) protocol, Handley brings substantial pedigree to the role: his work on Broadcom's AI networking architecture helped establish the foundations for the Ultra Ethernet Consortium (UEC).

"Ethernet was always likely to be a big player, with the scale of the whole industry driving forward a common, shared technology," Handley said. "And as a transport protocol, we want MRC to be in the same space for its niche." MRC is built on remote direct memory access (RDMA) over converged Ethernet (RoCE) and 800 Gigabit-per-second (800G) Ethernet interfaces.

Officially revealed by OpenAI earlier this year, the transport protocol accelerates massive-scale AI model training with greater GPU efficiency and reliability. The protocol was created with support from AMD, Broadcom, Intel, Microsoft, and Nvidia, with a shared goal to make huge GPU fleets behave like one tightly synchronized machine, even when network failures occur.

"Synchronous pre-training, the first stage of AI training, is the most compute-intensive piece," Handley explained. "When you're doing this kind of AI training, everybody starts talking at exactly the same time, which is the worst possible workload you can put onto a network, because there's no degree of statistical multiplexing."

Traditionally, systems rely on RDMA where each communication takes one pass through the network and remains pinned to a single path. This approach creates a collision risk: when pseudorandomly selected paths happen to coincide on one link, operations slow dramatically. "If 10 of them collide, then each of them is going at a tenth of the speed," Handley said.

MRC overcomes this bottleneck by breaking each path into many small packets, each taking a different route through the network. When every transfer is sprayed across hundreds of paths, "you end up load balancing the network beautifully smoothly" by avoiding congestion in the core. The protocol also tracks conditions per path, avoiding congested routes until they recover. "If one of those links fails, it's a very small impact on that flow," Handley noted. "Any network failure that happens above the first top switches is basically not consequential to us."

Beyond traffic patterns, MRC fundamentally changes the network control plane. Normally, switches run dynamic routing protocols to determine which links are operational and route around failures. But with an endpoint-driven system that sprays packets across many paths and avoids failures autonomously, routers no longer need to communicate with one another. IPv6 segment routing helps endpoints select exact paths, with the network adapter instructing packets precisely where to go.

"All the switches will do is blindly follow their orders," Handley said. "That moves a whole set of complexity out of our networks that we don't need anymore. Usually as you scale up, you care more and more about the network being smart in routing around failure. And actually, we did exactly the opposite, and made the network dumb. It doesn't even try to route around failures, because MRC can route around them for us."

These self-healing properties fundamentally change how OpenAI coordinates with infrastructure partners such as Oracle and Microsoft. Because MRC can detect failed or degraded paths within microseconds and stop using them immediately, network maintenance no longer requires tight synchronization with running training jobs or constant human vigilance. "Previously, we needed to coordinate with the people running the jobs in order to say, is this a good time to reboot a switch?" Handley said. "Now it says you can just reboot a switch. Doesn't matter. MRC won't care. So this gives us a whole bunch more flexibility in terms of operating these really huge networks with small teams."

MRC also reduces hardware complexity by splitting 800G interfaces into smaller links, allowing one interface to connect to eight different switches and creating eight parallel 100G planes instead of a single network. Rather than linking 64 high-speed connections, the system can link 512 smaller ones. This means more than 130,000 GPUs can be connected using only two layers of switches instead of the traditional three or four, reducing power consumption and the number of switches and optical components needed. "There's actually less stuff to fail, which helps," Handley explained.

"It also gives me more GPUs connected with only one switch in the middle, which gives me lower latency," he added. "But the thing I really care about is now from the same GPU, I have eight immediate paths. And so if I lose a link that connects directly to the GPU, I still have seven paths available. We stop using that particular path, but we can recover those packets really quickly, and we're back up to full speed very fast. This basically means that we no longer really have to worry about failures."

Today, MRC is deployed across all of OpenAI's largest supercomputers, including the Oracle Cloud Infrastructure-built supercomputer and Microsoft's Fairwater supercomputers. Infrastructure vendors including Nokia, Hewlett Packard Enterprise, and Arista Networks tout MRC's benefits. Dell'Oro Group research predicts Ethernet will generate nearly $80 billion in data center switch sales by 2030. Handley promised that future generations and ongoing collaboration with OpenAI partners will "keep rolling this ball forward" amid the ongoing AI buildout. "I've worked through a lot of exciting changes on the internet over the years, but the crazy scale of the things we're trying to do right now is really difficult, and coming up with the right way of solving them is a huge challenge. It's a lot of fun," he said.