Broadcom presented detailed performance metrics for its Thor Ultra 800GbE NIC at Hot Chips 2026, highlighting its role in next-generation AI cluster networking.

Broadcom took the stage at Hot Chips 2026 to present the Thor Ultra Ethernet NIC, an 800GbE adapter engineered specifically for AI and high-performance computing (HPC) fabrics. Following the family’s initial launch earlier this year, this session provided a detailed examination of its underlying architecture. Broadcom positioned the Thor Ultra within a comprehensive Ethernet strategy that spans scale-up within the rack, scale-out across racks, and scale-across between data centers. Each tier demands a unified set of NIC capabilities: high bandwidth, enhanced Remote Direct Memory Access (RDMA), low latency, quality of service (QoS), alongside virtualization, security, precise timing, telemetry, and diagnostics. The presentation preceded NVIDIA’s scheduled framework overview later in the same session block, establishing a natural comparative context for attendees.

The Thor Ultra couples a PCIe Gen6 x16 host interface with an eight-port 100G MAC, 256 SR-IOV virtual functions, and eight 100G SerDes lanes. It hardware-offloads RoCEv2, stateless packet processing, and inline encryption. Its enhanced RoCE implementation supports over 64,000 queue pairs, integrated with a TruFlow engine and BroadSAFE security framework. Broadcom highlighted several generational advancements over the previous Thor architecture. Key innovations include packet spraying with out-of-order placement, reliability mechanisms surpassing traditional Go-Back-N protocols, simplified programmable congestion control, and doubled port density enabled by multiplane connectivity.

The silicon is fabricated on a 5nm process, featuring a 2.4-billion-transistor count housed in a 27×27 mm package with a maximum power draw of 40–42W. Visible on the die are transmit and receive buffers, an embedded CPU subsystem, and P4-like programmable engines responsible for much of the enhanced RoCE workload. As noted during the demonstration, the die shot underscores the unprecedented complexity integrated into modern networking silicon. Broadcom offers the Thor Ultra in both OCP NIC 3.0 and standard PCIe form factors, configurable with either a single OSFP112 connector or dual QSFP112 connectors. The OSFP112 interface supports flexible lane configurations: 800G, dual 400G, quad 200G, or octal 100G, allowing operators to deploy a single board design across varying port densities. Supported cabling includes DAC, LPO, AEC, and optical transceivers, with the board consuming up to 50–55W excluding optics.

The packet pipeline architecture divides responsibilities between firmware/PCIe-managed threads and programmable engines. Transmit and receive threads manage work queues, transport and congestion control, programmable scheduling, and plane management. Meanwhile, the P4-like programmable engines execute customizable packet processing tasks. The enhanced RoCE feature set leverages multipath routing to spray packets across up to eight distinct planes. Out-of-order placement ensures RDMA responses are processed efficiently while maintaining strict ordering for sends and atomic operations. Reliable delivery is achieved through selective acknowledgment (SACK) and negative acknowledgment (NACK) mechanisms paired with automatic retransmission. Per-packet path selection utilizes headers containing placement metadata. Selective acknowledgment bitmaps track received packets per queue pair, enabling the receiver to reorder segments prior to delivering messages in sequence.

Congestion control is anchored by Receiver Credit-Based Congestion Control as the baseline for eRoCE. The receiver distributes credits to active senders, employs speculative credits for rapid line-rate initialization, and monitors ECN, traffic trimming, and CSIG telemetry. Customizable algorithms are implemented via P4-like match-action engines. The architecture utilizes ‘P4-like’ match-action engines rather than native P4, a distinction noted during the briefing. For GPU and XPU architectures, the Thor Ultra implements a peer memory pathway utilizing `dma-buf` for local peer-to-peer DMA between the NIC and GPU memory, while eRoCE handles remote data transfers. The software stack places the RoCE user library in user space and the RoCE driver in the kernel. Multi-tenancy capabilities incorporate standard modern NIC features, including granular QoS policies and robust virtualization support.

Precision timing features support IEEE 1588 PTPv2 with nanosecond-level packet timestamping and Precision Time Measurement. Pulse-per-second (PPS) input/output, a dedicated time-of-day plane, and an external TCXO input synchronize the Physical Hardware Clock (PHC), ensuring tight synchronization across AI and HPC systems. For further context on network timing implementations, Slicast recently captured footage at a major hyper-scaler’s networking lab, with a dedicated analysis scheduled for release in early September. Hardware security begins at the silicon level with a FUSE-based root of trust, a ROM bootstrap loader, and authenticated firmware updates. Device attestation via SPDM and PSP-based encryption/decryption complete the security architecture. Comprehensive health monitoring encompasses chip and optical component temperature tracking, eye diagram analysis, system diagnostics, crash and core dump collection, and standards-compliant manageability aligned with DMTF and OCP specifications.

The Linux kernel stack separates functionality into the `bng_re` RoCE driver and the `bng_en` NIC driver. Above these sits the `libbng_re` user-space library, all built atop standard `libibverbs` and `rdma-core` interfaces. Peer memory and direct collective operations integrate with NCCL, RCCL, and MPI through a verbs provider and xCCL plugins. This architecture relies on an upstream `dma-buf` peer memory model, ensuring application code remains unmodified. Independent validation on platforms such as the Keysight CyPerf bench—mirroring prior testing of the NVIDIA ConnectX-8 C8240 800G Dual 400G NIC, which achieved 800 Gbps on a PCIe Gen5 server—represents a logical next step for third-party verification.

Broadcom presented initial collective communication benchmarks on a Gen5 GPU platform. Testing across two nodes—each equipped with eight Gen5 GPUs and 16 ranks communicating over sixteen 400G links—yielded an `all_reduce` throughput of 383.93 GB/s and a `reduce_scatter` throughput of 380.23 GB/s, approaching the theoretical 400 GB/s ceiling. The `alltoall` operation recorded 84.62 GB/s, a result consistent with its inherently higher communication overhead. Overall bus bandwidth exceeded 96% of line rate across most benchmarked operations. TCP performance was measured using iperf3 on a Gen6 CPU platform equipped with 512 GB of DDR5-8000 memory. Unidirectional throughput reached 791 Gbps across 16 parallel flows, representing 98.9% of the 800G link capacity. Bidirectional traffic aggregated to approximately 1.51 Tbps, with throughput scaling nearly linearly up to eight flows before link saturation. RDMA write benchmarks show unidirectional throughput saturating at 781 Gbps, while bidirectional throughput reaches 1,558 Gbps—approximately 97.6% of the 1.6 Tb/s aggregate capacity and roughly double the unidirectional rate. At 32 KB message sizes, unidirectional performance achieves approximately 88% of line rate, narrowing to within 3% of peak throughput at 128 KB.

Broadcom emphasized that the architectural challenges addressed in this generation—including multipathing, RoCE enhancements such as reliability and out-of-order placement, and programmable congestion control—are fundamentally rooted in the packet processing pipeline. The presentation covered the Thor Ultra’s full stack, from the physical silicon die to the Linux software toolchain. The architecture is explicitly designed as a systems-level solution for AI scale-out, demonstrating cohesive integration across silicon, software, and manageability frameworks. With the Thor Ultra launch, Broadcom positions itself directly in the competitive 800G NIC market alongside NVIDIA’s ConnectX-8 and AMD’s Vulcano. The data path is optimized for enhanced RoCE or MRC++ protocols, while integrated QoS and virtualization capabilities facilitate XPU and CPU-as-a-Service deployments. Comprehensive security, manageability, and precision timing ensure robust synchronization across distributed systems. Following initial hardware previews at OCP Summit 2025 nearly a year ago, this session delivered highly anticipated architectural deep dives into the Thor Ultra’s internal design and performance characteristics.