Stanford researchers are promoting Homa, a new network protocol designed to replace TCP by rethinking latency and throughput optimizations for AI-scale cluster applications.
TCP, the foundational data transport protocol underlying most of the internet and cloud computing, is fundamentally ill-suited for modern AI workloads, according to John Ousterhout, professor emeritus of computer science at Stanford University. To address this gap, he is championing Homa, a new protocol designed specifically for datacenter environments.
Despite TCP's ubiquity, integrating Homa into existing networks requires minimal effort. The protocol can be compiled from its open-source repository and installed as a module in Linux kernels on clients and servers—no reboot necessary. Critically, Homa operates alongside TCP, enabling gradual migration of applications without requiring wholesale infrastructure replacement. Installing Homa actually accelerates remaining TCP traffic as well.
Homa represents a clean-slate redesign of network traffic management. Work on the protocol began as a PhD dissertation first published in 2019 by Behnam Montazeri, now a Google staff engineer. Since retiring from teaching, Ousterhout has made propagating Homa his "life's mission."
The key difference between Homa and TCP lies in their foundational architecture. TCP is stream-based; Homa is message-based, with explicit message lengths similar to remote procedure calls. Crucially, Homa delegates congestion control to the receiver rather than the sender. When the receiver obtains the first packet, it learns the total incoming data size and can explicitly schedule subsequent packet transmission. This approach prioritizes shorter messages using a shortest-remaining-processing-time algorithm, reducing latency for brief messages by an order of magnitude. At the 99th percentile, Homa achieves 92 microseconds latency for shorter messages—thirteen times faster than TCP's 1.2 milliseconds p99 latency on a 100 Gbps network at 80% utilization. Even for longer messages, Homa demonstrates a twofold latency advantage.
Ousterhout is currently drafting an IETF standardization document and working to upstream Homa into the Linux kernel. In March, the protocol was backported to Red Hat Enterprise Linux 8 and 9.5. He is also collaborating with large financial services firms to prototype the technology.
Not all network architects accept the case for replacing TCP. In 2023, prominent network architect Ivan Pepelnjak published a critical position paper questioning Ousterhout's performance claims and arguing that Homa represented a solution in search of a problem.
TCP's limitations extend beyond AI infrastructure. The high-performance database community uses DPDK (Data Plane Development Kit) to bypass the TCP stack; storage networks have adopted NVMe-oF with RDMA and Fibre Channel transports; Google developed QUIC—now the foundation for HTTP/3—to eliminate TCP's head-of-line blocking; high-frequency trading and multiplayer gaming communities have long suffered from TCP latency; and specialized RDMA fabrics and Amazon's Scalable Reliable Datagram have emerged as alternative approaches.
TCP's constraints stem from its byte-stream model. Vint Cerf and his contemporaries designed TCP to impose order on network traffic, providing flow control, guaranteed delivery, connection handshakes, and congestion control. Yet this design conflates two separate problems: congestion control, which prevents network saturation, and flow control, which prevents senders from overwhelming receivers. Without message boundaries, TCP treats all data as an undifferentiated byte stream. Servers can signal senders to reduce input, but lack visibility into remaining in-flight traffic. Senders must guess how aggressively to throttle based on acknowledgment timing.
For general internet traffic or large datacenter transfers, modest latency increases are acceptable. For latency-sensitive AI workloads, even millisecond delays create significant cost. Large language model training requires sustained high-bandwidth transfers of weight gradients, model weights, KV cache entries, and checkpoints. Yet these bulk transfers must now coexist with short bursts from agents and control tasks such as metadata coordination and cache lookups. Each millisecond of latency idles expensive GPUs—a consequence Ousterhout describes as untenable. "Legacy protocols are poorly suited for this environment," he noted.
For now, TCP remains the dominant protocol, though its obsolescence for datacenter workloads grows increasingly apparent.