OpenAI’s first custom accelerator, Jalapeño, is designed to address memory bandwidth bottlenecks, mirroring efforts by Moonshot AI’s Kimi K3.

OpenAI has released its first comprehensive performance data for Jalapeño, a custom inference accelerator developed in partnership with Broadcom. The figures reveal a chip engineered around a single core challenge: delivering data to compute cores quickly enough to eliminate idle time. Benchmarked using SemiAnalysis’s InferenceX suite across three model families—GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T—Jalapeño achieved 1.5 to 1.9 times greater mixed-token throughput per kilowatt at peak operation and reduced end-to-end request latency by 1.7 to 3.6 times compared to the leading commercial systems OpenAI evaluated. For highly latency-sensitive, interactive workloads, this performance advantage expanded to 2.1 to 4.1 times.

The most striking metrics emerge when decode speeds are matched precisely to the maximum capability of the comparison hardware, after which throughput per kilowatt is measured. At that equilibrium, Jalapeño generated 53.7 times more mixed tokens per kilowatt than the reference system on GPT-OSS, 104.3 times more on DeepSeek R1, and 56.1 times more on Kimi K2.5. These multipliers highlight the widening advantage at peak, high-demand operating points. OpenAI’s own charts indicate the performance gap narrows significantly under lighter workloads, meaning these figures represent an extreme-case ceiling rather than a typical operational baseline.

OpenAI evaluated performance through the lens of actual operator experience: measuring useful work completed per unit of power while maintaining the strict latency requirements of interactive agents, rather than prioritizing raw throughput alone. This distinction is critical because agentic workflows chain numerous model calls together, causing minor per-step delays to compound across entire tasks. To normalize results, OpenAI used each accelerator’s published power rating—Jalapeño at 700 watts versus comparison chips rated at 1,200 and 1,400 watts—and confirmed that Jalapeño’s measured sustained power remained at or below 550 watts across all tested workloads. Against the largest model in the benchmark, Kimi K2.5 (one trillion parameters), Jalapeño delivered approximately 1.5 times higher peak performance per watt and 3.4 times lower end-to-end latency than the reference system. While this margin is narrower than on the other two models, it remains sufficient to place Jalapeño on what OpenAI terms the Pareto frontier for the tested operating range.

Jalapeño’s architecture is grounded in a precise observation about language model execution: prompt processing is compute-bound, while autoregressive token generation is primarily constrained by memory bandwidth. Inter-chip communication delays further exacerbate the issue, leaving compute units idle while awaiting data. To mitigate this, OpenAI designed Jalapeño to minimize data movement, ensuring model state—including the key-value cache required during generation—is explicitly localized. A broad networking fabric built on Broadcom’s Tomahawk technology enables entire workloads to remain within a single interconnected system, eliminating the need to shuttle data across chip boundaries. Tom’s Hardware’s analysis of publicly released chip imagery corroborates this memory-centric design: the packaging features a single large compute chiplet encircled by six HBM modules, a configuration optimized for localized memory access rather than pure compute density.

Broadcom’s involvement extended well beyond manufacturing. The company supplied the silicon implementation and networking fabric, while Celestica managed board, rack, and system integration. OpenAI has positioned this collaboration as a replicable model: AI operators with deep expertise in their own models and serving patterns can partner with seasoned silicon firms instead of investing in internal fabrication capacity. Broadcom CEO Hock Tan has stated that compute demand across the company’s AI customer base remains effectively limitless through the end of the decade. Similarly, OpenAI President Greg Brockman told CNBC that the company “cannot get compute fast enough,” a sentiment that aligns with OpenAI’s strategy of continuous expansion rather than substituting one supplier for another.

Jalapeño progressed from initial design to tapeout in just nine months, a pace OpenAI partially credits to leveraging its own models to explore chip implementations and accelerate verification loops, including optimizations for the chip’s arithmetic circuits. This methodology extended beyond hardware: OpenAI designed Jalapeño as a deterministic programming target, requiring engineers to define workloads as local tensors with explicit communication protocols. This shifts the complexity to AI-driven systems, which handle the difficult task of scheduling and mapping operations across the full infrastructure. By utilizing Codex with GPT-Astra, the engineering team optimized three open-weight models outside Jalapeño’s original production scope to high performance within two months. For specific attention and mixture-of-experts blocks in GPT-OSS, AI-generated implementations executed 1.5 to 1.8 times faster than their human-authored counterparts. This represents a microcosm of a broader trend in frontier AI development: capabilities first validated in constrained, verifiable environments—such as chip design and kernel optimization—gradually scale into production systems once the foundational approach is proven.

Five weeks prior to OpenAI’s Jalapeño announcement, Moonshot AI unveiled Kimi K3, a 2.8-trillion-parameter model that activates only 104 billion parameters per token across 896 experts. Market reaction mirrored the seismic shift triggered by DeepSeek’s early 2025 debut. Semiconductor equities declined following the news, with Bloomberg reporting sharp sell-offs among Chinese AI competitors; Z.ai fell 28 percent and MiniMax dropped 16 percent on the same day. However, Bloomberg’s subsequent analysis contended that comparing Kimi K3 directly to DeepSeek was misleading. While DeepSeek’s breakthrough focused on reducing training and inference costs, Kimi K3 enhances computational efficiency at a scale that drastically increases memory infrastructure demands. Rather than undermining memory markets, this architectural choice reinforces the strategic value of high-bandwidth memory solutions from SK Hynix and next-generation systems from Nvidia.

Moonshot’s technical documentation aligns with this assessment. Kimi K3 is built around Kimi Delta Attention, a hybrid linear attention mechanism designed to alleviate the memory bottlenecks inherent in standard transformer architectures as context windows expand. This is complemented by a novel inter-layer data movement technique Moonshot terms Attention Residuals. Independent analyses of the model’s hosting requirements identify decoding as the primary bottleneck: since each newly generated token accesses the model’s weights exactly once, compute cores remain largely starved for data, waiting on high-bandwidth memory regardless of routing sparsity. Moonshot’s official deployment guidelines recommend supernode configurations comprising 64 or more accelerators for production environments. Furthermore, third-party assessments of the released checkpoint estimate a minimum practical VRAM footprint exceeding 1.6 terabytes.

China’s memory supply chain remains the critical bottleneck for this architecture. While Huawei has spearheaded a domestic consortium aimed at developing high-bandwidth memory (HBM) chips, advanced HBM manufacturing remains heavily concentrated among SK Hynix, Samsung, and Micron. China’s most advanced domestic memory producer has yet to demonstrate the ability to fabricate HBM at the densities required by modern accelerators, let alone achieve volume production. A Bank of America research note accompanying Kimi K3’s release suggested that large-scale pretraining combined with architectural efficiency could still yield substantial performance gains for Chinese laboratories despite ongoing compute restrictions. However, this perspective functions more as a mitigation strategy for memory scarcity than a fundamental resolution.

Viewed side by side, Jalapeño and Kimi K3 illustrate the same underlying industry pressure from divergent starting points. OpenAI operates with near-unrestricted access to cutting-edge silicon and pursues custom chip development to maximize power efficiency and cost optimization—a strategy where even a 1.5x improvement in tokens per kilowatt translates into substantial financial savings across massive data center fleets. Conversely, Moonshot navigates export restrictions limiting access to next-generation accelerators and associated memory infrastructure, responding by architecting a model that inherently demands less memory bandwidth per token rather than relying on future hardware availability. Both approaches converge on a singular conclusion: as model scales expand and inference workloads evolve toward longer, more agentic tasks, memory bandwidth has emerged as the critical resource that AI laboratories across the Pacific are now prioritizing over raw compute throughput.

OpenAI states it plans to begin deploying Jalapeño within its own infrastructure by the end of 2026, while continuing to expand its use of Nvidia and other suppliers rather than replace them. The company describes this hybrid procurement strategy as essential to sustaining growth velocity without overcommitting to single-vendor dependencies.