VentureBeat argues that idle GPUs are not stopping AI platform plans from moving ahead.

Most surveyed companies running AI in production say the GPUs they use sit idle at least half the time, and most of those companies plan to add or switch AI compute platforms anyway, according to the latest data from VentureBeat Intelligence.

A whopping 79% of the 218 respondents who reported GPU utilization said their GPUs are busy half the time or less, across three monthly VB Pulse surveys from July through September. Among those companies, 70% said they plan to adopt a new, additional or replacement AI compute platform within the next 12 months. The share with idle GPUs held between 77% and 81% in each of the three months. We reported the September results Thursday; the three months of data show the pattern holding steady.

Those numbers matter because Wall Street is arguing over whether AI is a bubble. On October 13, the International Monetary Fund warned that a sharp fall in AI share prices could threaten global financial stability. Most of that argument has focused on the companies selling AI compute, such as Nvidia, cloud giants Microsoft, Google and Amazon Web Services, and specialized GPU clouds like CoreWeave, and on whether they are building more capacity than their customers will pay for. We looked at what those customers are actually doing.

**The bubble is inside the enterprise**

At first glance, the numbers look like a contradiction. Why would companies with so much unused GPU capacity plan to add or switch AI compute platforms? And why would they leave GPUs idle when, as technology leaders tell me, they are under heavy pressure in this fall's budget season to cut their AI bills?

The demand for AI is real, but it is landing in a different place than the idle GPUs. Professionals and consumers keep using more AI, and much of that work runs through the frontier models from Anthropic and OpenAI. That demand will keep supporting those companies' spending on compute for some time. The mismatch sits one level down, inside enterprises that are paying for two kinds of AI that cannot easily stand in for each other.

The first kind is capacity. I believe many companies were scared by reports of GPU shortages in 2024 and 2025 and lined up GPU capacity so they would not be caught short. Much of the capacity companies use sits idle, our surveys show, though the surveys did not ask how companies got that capacity or whether they controlled it directly.

The second kind is frontier models. Engineers and other employees have preferred the most capable models from Anthropic, OpenAI and Google, paid by the token through those companies' APIs, through monthly subscriptions or through the big cloud providers. The thinking was that companies needed the best intelligence to race ahead, and they would sort out the economics later. Uber's chief technology officer, Praveen Neppalli Naga, said in August that "we're coming to the end of the so-called tokenmaxxing era," after Uber spent its 2026 AI budget within a few months, according to Fortune. An AI leader at a Fortune 500 company, speaking on background, told me in October that AI costs keep rising, and that while the company expects productivity gains to offset them eventually, it does not know that yet.

Amazon Web Services, Microsoft and Google now sell Anthropic's or OpenAI's models through their clouds, and OpenAI said on Wednesday that its models will reach Google Cloud in the coming weeks. OpenAI's frontier GPT models and Anthropic's Claude models are closed, so the cloud provider or the model maker runs them on its own infrastructure and bills by the token or for capacity reserved for that one model. A company cannot load them onto GPUs it already controls. To shift work onto those GPUs, a company has to switch to an open-weight model and run it itself. That is real engineering work, and the cheapest options raise questions of their own. In VentureBeat's own tests in October, DeepSeek was the cheapest of five open models we ran, but DeepSeek comes from a Chinese lab, and many companies are still reluctant to double down on Chinese models as their main workhorse models, for a variety of reasons.

Our data fits this reading in three ways:

- **Companies with idle GPUs are still planning new platforms.** Within the next year, 70% of companies with idle GPUs plan to adopt a new, additional or replacement AI compute platform. That is about the same as the 74% of companies whose GPUs are busy, a statistically insignificant difference.

- **Companies with idle GPUs are considering cloud platforms or model providers.** Among those planning a new platform, 90% named a cloud platform or a model provider among their options, and only 8% named an on-premises or colocated GPU cluster or a self-managed open-source stack.

- **Companies that run their own stack report higher GPU utilization.** Among companies running AI on an on-premises or colocated GPU cluster or a self-managed open-source stack, 56% said their GPUs sit idle at least half the time, against 83% of all other companies running AI in production. We did not ask why. My guess is that a team running its own stack sees every idle hour directly and has the engineers to fill it.

Few companies planning a new AI compute platform have a finance-reviewed ROI figure. In the September survey, 91% of all companies planning a new AI compute platform had no return-on-investment figure their finance team had reviewed. To be sure, some idle capacity may be deliberate, since a company may reserve GPUs for spikes in demand, and a planned platform may replace an older one.

**The next savings are in tokens, context and agents**

I expect a growing share of companies' AI spending to go to model usage. By this, I mean the per-token charges that model makers and cloud providers collect on every request. Those charges are likely to grow because companies are shifting to AI agents, which plan and carry out multi-step tasks. Inference costs per agentic workflow will rise more than fivefold through 2028, Gartner has predicted. Inference is the work a model does to answer each request. The price of each token is falling, but agents use so many more tokens, and more expensive ones, that the falling prices do not keep up.

Most of the tokens a coding agent processes are context, meaning the instructions, documents and history it rereads at every step. That is why caching matters so much. In a technical guide first published in March, two Nvidia engineers reported that after the first call in a Claude Code session, later calls reuse 85% to 97% of their context from a cache. Reusing that context is far cheaper than processing it fresh: Anthropic charges between one-fortieth and one-tenth of its normal input price for cached tokens, depending on the model.

Adding agents multiplies the bill, too. Coding-agent teams, with a lead agent running up to five subagents at once, cost 1.8 to 5.1 times as much as a single agent, and only one of four team setups improved benchmark scores by a statistically significant margin, according to Vals AI.

Companies should get more finished work out of the compute and tokens they already pay for before they sign the next contract. In practice, that means six things:

- measuring the cost of each completed task, across both GPU capacity and API spending;

- testing which model and effort level to set as the company default, judged by the cost of each finished task rather than the price per token, since a cheaper model can use so many more tokens that it costs more per task;

- sending each agent only the context it needs, and caching repeated context, whether through model-provider APIs or on companies' own infrastructure;

- adding a second agent only when the task splits into parallel pieces;

- moving routine, high-volume work, such as classification, extraction and summarization, from frontier model APIs to open models running on idle capacity;

- filling the remaining idle GPU hours with work that can wait, such as batch inference, model evaluations and fine-tuning.

**VentureBeat's October 28 summit takes on these choices**

Those choices are the agenda of VentureBeat's Agentic Infrastructure Stack Summit on October 28 in San Francisco, under the theme "Smarter Agents Need More Memory. The Best Teams Will Use Less." The day runs in four parts:

- **How many agents to run:** Jeff Simon, chief information officer of T-Mobile; Val Bercovici, chief AI officer of WEKA; Deepika Khera, senior director of AI at PayPal; and Ben Kus, chief technology officer of Box, on coordinating agents while controlling cost and risk.

- **How much context to give them:** Anjul Bhambhri, senior vice president of engineering at Adobe, and Vijoy Pandey, senior vice president and general manager of Outshift by Cisco, on memory, context and coordination.

- **What inference really costs:** Micah Hill-Smith, co-founder and CEO of Artificial Analysis, and Uday Kiran Medisetty, distinguished engineer at Uber, on inference economics, model routing and the cost of completed tasks. The track also includes