OpenAI’s proprietary Jalapeño accelerator delivered strong inference benchmarks, validating its custom silicon design against industry standards.
OpenAI’s first custom AI accelerator, Jalapeño, has entered public benchmarking, offering the industry its clearest view yet of how the company intends to cut costs and latency for large-scale model inference. Designed specifically for large-language-model inference, the first-generation chip’s headline performance comparisons, however, remain tied to SemiAnalysis’s tested workloads and system configurations.
Developed by OpenAI in partnership with Broadcom, with Celestica providing board, rack, and system integration expertise, Jalapeño was unveiled in June as the inaugural product of a multi-generation computing platform slated for deployment by late 2026. New results from SemiAnalysis’s InferenceX benchmark evaluate Jalapeño against commercial accelerator systems running GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. According to OpenAI, the chip achieved higher peak mixed-token throughput per kilowatt and reduced token latency compared to the reference systems.
Detailed figures and charts were reproduced in a report by ifanr. In a DeepSeek R1 test featuring an 8K-token input and 1K-token output, end-to-end latency dropped from 5.99 seconds on the comparison system to 1.65 seconds on Jalapeño. Similarly, the minimum interval between generated tokens decreased from 5.90ms to 1.43ms.
These findings should not be extrapolated to suggest that Jalapeño outperforms every GPU across all workloads. Inference performance fluctuates based on model architecture, batch size, quantisation, latency targets, software stack, rack design, and power accounting methodologies. SemiAnalysis conducted the testing alongside OpenAI engineers, meaning the comparisons strictly reflect the specific hardware and configurations disclosed by the benchmark. OpenAI’s own analysis of the measured results mirrors this cautious framing, highlighting improved throughput and latency on the tested models without making unrestricted performance claims.
Unlike training-focused chips, Jalapeño is built exclusively for inference. LLM inference typically cycles between a compute-intensive prefill phase, which processes the initial prompt, and a decode phase that continuously shifts model weights and key-value cache data while generating tokens. To address this, OpenAI engineered the architecture to minimise data movement and optimise the balance between compute, memory, and networking resources, aiming to capture more of the chip’s theoretical performance in real-world deployments. SemiAnalysis reports the chip features 216GiB of HBM4 memory and 15.4TB/s of memory bandwidth, though these detailed specifications originate from the research firm rather than OpenAI’s initial launch materials.
The architecture partitions compute and memory resources to keep model data proximate to the cores processing it, relying on high-speed interconnects only when data must traverse different regions. This design philosophy prioritises reducing the energy and time traditionally spent shuttling data between compute units, memory, and adjacent accelerators. Power metrics reported by SemiAnalysis indicate a 700W nominal envelope, with sustained draw remaining under 550W during testing. Readers should note that performance-per-watt comparisons vary depending on whether power is measured at the chip, board, server, or facility level; evaluating the benchmark’s charts requires adhering to its stated “all-in utility” methodology.
OpenAI states that Jalapeño progressed from initial design to manufacturing tape-out in just nine months. The company credits this accelerated timeline to tight software-hardware co-development, Broadcom’s implementation expertise, and the internal use of OpenAI models during select phases of design exploration, verification, and optimisation. It is important to clarify that this nine-month window encompasses only the design-to-tape-out phase, excluding later stages such as fabrication, packaging, bring-up, validation, software enablement, and production qualification.
According to ifanr’s timeline reconstruction, OpenAI received first silicon in 2026 and dedicated several months to hardware bring-up and model adaptation. The publication notes that Codex and an unreleased internal model were leveraged to port workloads outside the accelerator’s original enablement scope. These specific timing and tooling details should be understood as third-party reporting that extends beyond OpenAI’s official communications.
The collaborative framework divides responsibilities clearly: OpenAI provides the accelerator architecture and deep expertise in model-serving workloads; Broadcom handles silicon implementation, networking, and connectivity technology, including Tomahawk networking components; and Celestica drives the industrialisation of boards, racks, and full systems. This partnership supports a broader initiative scaling up to 10 gigawatts of OpenAI-designed accelerators and networking infrastructure. Per Broadcom’s 2025 announcement, initial rack deployments are scheduled to commence in the second half of 2026 and extend through 2029.
Strategically, Jalapeño gives OpenAI greater vertical control over the inference stack. A processor tailored to its specific serving patterns promises lower operational costs, expanded capacity, and faster response times for services like ChatGPT, Codex, and the developer API. Realising these advantages, however, hinges on production yields, system reliability, software maturity, and the degree to which controlled benchmark results translate to live, scaled deployments.
Consequently, Jalapeño’s early benchmarks are highly significant but not definitive. They prove that a first-generation custom accelerator can compete in rigorous inference scenarios. Broader industry conclusions will ultimately depend on reproducible testing across a wider range of models, latency targets, and fully integrated data-centre environments.
Sources: OpenAI’s Jalapeño announcement; OpenAI benchmark commentary; SemiAnalysis InferenceX; Broadcom; and ifanr.