OpenAI has published the first performance results for Jalapeño, its custom chip designed to run AI models rather than train them. The company says the accelerator delivered higher throughput per unit of power and lower response latency than currently available systems in tests conducted with SemiAnalysis’ InferenceX benchmark.

The results, presented at the Hot Chips conference on Tuesday, point to OpenAI’s broader effort to control more of the infrastructure behind its products. But the comparison comes with important limitations: OpenAI supplied the benchmark figures, some runs were verified by SemiAnalysis on-site, and Jalapeño is not expected to enter even small-scale deployment until late 2026. More substantial use is planned for 2027.

Technical design and benchmark evidence

Jalapeño is an inference accelerator, meaning it is intended to generate responses from trained models. Unlike general-purpose systems used across both training and inference, the chip is designed around the delays and data movement associated with serving large language models.

OpenAI says the system keeps model state, including the KV cache used during response generation, close to the relevant compute and memory resources. It also targets prefill and communication stages that can slow down inference, particularly when models must handle long prompts or serve many users at once.

The company reported between 1.5 and 1.9 times more AI work per watt at peak throughput across three tested models. It also claimed end-to-end latency improvements ranging from 1.7 to 3.6 times versus the best commercially available systems in the comparison. For interactive workloads, OpenAI reported performance gains between 2.1 and 4.1 times.

The tested models were GPT-OSS 120B, Deepseek R1 670B and Kimi K2.5 1T. According to The Decoder’s account of the presentation, Jalapeño reached roughly 1,400 tokens per second per user on GPT-OSS and more than 700 tokens per second on Deepseek R1 with a single concurrent request.

Those figures are not a universal ranking of AI hardware. OpenAI provided the numbers used in the InferenceX results, while SemiAnalysis verified some runs in its laboratory. The comparison systems also used optimization techniques, including multi-token prediction and speculative decoding, that Jalapeño did not use in the reported tests. That could leave room for improvement, but it also makes the results harder to compare without examining the complete configurations and workloads.

Nvidia comparisons are significant but provisional

The headline comparison was against an Nvidia Blackwell system, which is currently widely deployed for AI workloads. The Decoder also reported that SemiAnalysis considered Nvidia’s newer Vera Rubin platform a more appropriate comparison because both systems use HBM4 memory. On that comparison, Jalapeño reportedly produced more output tokens per megawatt, although the two platforms were roughly even on total cost of ownership per token.

The timing matters. Vera Rubin systems are already shipping to customers, while Jalapeño reportedly remains at the engineering-sample stage. Nvidia and AMD have also published results on larger or newer models, including DeepSeek V4 Pro and Kimi K3, that were not included in the Jalapeño tests cited here.

That makes OpenAI’s performance advantage a promising early result rather than a settled market position. Hardware benchmarks can change substantially when models, kernels, memory configurations, software stacks and serving policies change. The eventual test will be how Jalapeño performs on the models and traffic patterns OpenAI operates at production scale.

A custom silicon strategy with Broadcom

OpenAI began developing Jalapeño with Broadcom in 2024, according to the reports. The final design was sent for fabrication in November 2025. OpenAI said the complete cycle took about 16 months, with nine months between the initial chip design and the finished blueprint being sent to the factory.

The company also said it used its own AI models during development. Older models assisted with chip design, while newer systems helped with programming and optimization. Those claims illustrate the intended feedback loop: models help build the hardware, the hardware serves the models, and production workloads provide data for further software and architecture decisions.

OpenAI is presenting Jalapeño as a multigenerational platform rather than a one-off processor. The approach coordinates chips, memory, networking and models around the phases of inference. OpenAI CFO Sarah Friar described this as part of a larger compute strategy linking data centers, models, products and devices.

The strategy does not necessarily mean OpenAI is abandoning external suppliers. The Decoder reported that the company views Jalapeño as complementary to relationships involving Nvidia, AMD, AWS, Cerebras, CoreWeave and other infrastructure providers. In practice, OpenAI’s custom hardware effort sits alongside those partnerships, while also giving the company another negotiating and capacity-planning option.

Implications for builders and enterprise buyers

For AI product teams, the most relevant promise is not a benchmark record by itself but the possibility of lower serving costs at high utilization. More output per watt could improve the economics of chat, coding assistants, voice systems and other products that generate large numbers of responses. Lower latency could also make longer-context applications feel more responsive.

The architecture’s focus on prefill, communication and local KV-cache placement is especially relevant to teams serving large models across many concurrent users. Those stages can become bottlenecks even when raw accelerator compute is available. A system designed around the complete inference path may therefore outperform a faster general-purpose accelerator on particular production patterns.

However, buyers should not treat the current results as evidence that Jalapeño is ready for broad external adoption. OpenAI has not indicated that the chip will be generally available as a commercial platform, and its initial deployment is expected to be small. Developers building on cloud infrastructure will still need to evaluate the systems that are available now, including their software compatibility, capacity, pricing and reliability.

The results do raise a competitive question for Nvidia’s software ecosystem. SemiAnalysis argued that the performance could weaken the company’s CUDA advantage, but that is a market interpretation, not a demonstrated change in developer behavior. CUDA compatibility, tooling, operator libraries, support and the ability to scale across suppliers remain important purchasing factors beyond tokens per second.

What to watch next

The first signal will be whether OpenAI meets its forecast for small-volume Jalapeño deployment at the end of 2026. Production availability, rather than engineering-sample performance, will reveal whether the design can be manufactured and operated reliably at scale.

The next benchmark cycle should include newer models such as DeepSeek V4 Pro and Kimi K3, larger concurrency levels and matched software optimizations across all systems. Independent testing will also be important for assessing power use, latency distributions, utilization and total cost per token.

Enterprise buyers should watch for details on access, pricing, supported model architectures and integration with OpenAI’s serving stack. For the wider market, the key question is whether OpenAI can repeat the result across later chip generations while Nvidia, AMD and other suppliers continue improving their own inference platforms.

Creati.ai perspective

Jalapeño is strategically important because it shows OpenAI treating inference economics as a product-design problem, not only a procurement decision. If its reported gains survive independent testing and production traffic, custom silicon could give OpenAI more control over latency, power consumption and capacity for its most heavily used services.

The evidence is still early and partly vendor-controlled. The strongest conclusion today is that OpenAI has produced a credible first-generation inference design with encouraging benchmark results—not that it has already displaced Nvidia or established a new hardware standard.