OpenAI’s custom Jalapeño inference accelerator is reshaping unit economics by significantly lowering per-token inference costs compared to standard GPUs.
OpenAI has released the first measured results for Jalapeño, its custom inference chip. Across three open-weight models, the company reports that Jalapeño delivers 1.5 to 1.9 times more AI work per watt at peak throughput and achieves 1.7 to 3.6 times lower end-to-end latency compared to reference systems. Independent reporting corroborates these headline figures while emphasizing two important caveats: the benchmarks were run against currently available hardware, and Jalapeño is designed exclusively for inference, not training.
The benchmark focuses on the user-facing portion of the workflow: inference, which encompasses processing a prompt and generating a response. OpenAI evaluated Jalapeño using SemiAnalysis’s public InferenceX framework, a tool designed to measure the complete model-serving pipeline rather than isolated arithmetic peaks. By capturing latency, throughput, and power consumption simultaneously, this methodology provides a more realistic picture of real-world performance—particularly for AI agents that rely on numerous sequential model calls.
According to OpenAI, Jalapeño achieved a superior balance of throughput and latency across GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. Specifically, the chip demonstrated approximately 1.5 times higher peak performance per watt and 3.4 times lower end-to-end latency on Kimi K2.5 1T relative to the comparison system. The true value extends beyond benchmark rankings: it lies in the ability to serve a higher volume of requests within the same power constraints.
TechCrunch notes that the comparison baseline included an Nvidia Blackwell system, while also cautioning that competing hardware may evolve significantly before Jalapeño enters broad deployment. OpenAI has not yet disclosed customer pricing, fleet allocation, or a timeline for when users might see measurable product-level improvements. Consequently, buyers should view these figures as credible technical validation rather than an indication of immediate cost reductions.
Vertical integration offers distinct advantages. When a model provider controls the entire serving stack, it can optimize memory placement, networking, and software specifically for actual workloads. OpenAI states that Jalapeño maintains model state locally and strategically balances the compute-intensive prefill phase with the memory-bound decode phase. Should these architectural gains hold during production deployment, OpenAI will gain substantially greater control over latency, capacity planning, and profit margins, reducing its reliance on the broader general-purpose accelerator market.
As Axios and TechCrunch have highlighted, Jalapeño is engineered strictly for inference, not for training frontier models. Both outlets characterize the chip as a strategic move to partially offset OpenAI’s reliance on Nvidia, rather than eliminate it entirely. For organizations assessing supply-chain concentration, it remains critical to distinguish between the silicon used to train models and the infrastructure required to serve them.