The recent reporting that Microsoft is rationing Nvidia Blackwell chips and Azure customers face wait times through the end of 2026 was about a supply crisis. Hyperscalers are increasingly prioritizing capacity, placing economic and operational pressure on smaller, emerging, and startup AI organizations. Companies are being forced into longer-term commitments at 30% and higher price increases just to access baseline capacities. One unnamed founder was told a tightly connected cluster of 1,000 graphics processing units (GPUs) would be nearly impossible to find at the largest providers.
What gets less attention is how enterprises use that GPU capacity when they have access to it.
In 2024 and into this year, the main enterprise AI approach has been to connect AI agents to data systems and expect that previously unavailable business intelligence would flow. Trickle perhaps, but flow, no. The State of AI in Business 2025 report from MIT revealed that most enterprise AI initiatives fail due to fragile workflows, a lack of contextual learning, and misalignment with day-to-day operations. Andreessen Horowitz (a16z) partners found the same in March: context is the root cause, not AI models and agents. The data environment in which they work is the constraint. Expensive GPU hours are wasted by agents trying to reason about unprepared data.
See also: How AI Is Forcing an IT Infrastructure Rethink
The cost of 5% utilization
Enterprise GPU workloads are at about 5% utilization, according to Cast AI’s 2026 State of Kubernetes Optimization Report, even as customers clamor for more capacity.
Hyperscaler landlords rent Blackwell chips at an average of $6 an hour. That unnamed founder’s 1,000-chip cluster costs over $140,000 a day. At 5% utilization, they’re paying frontier-chip prices for throughput that a fraction of that hardware, fed better data, could replicate.
Research indicates that four in five enterprises are exceeding their AI infrastructure budgets by more than 25%, and 84% say AI workloads are eroding gross margins.
The disconnect between AI adoption and business outcomes is represented by data inefficiency.
The AI infrastructure debt that doesn’t appear on the balance sheet
A modern AI progresses through CPU-heavy data loading, GPU-intensive inference or training, and back to the CPU for post-processing. When those stages share a container, the GPU is allocated across the full lifecycle, but it performs on only a fraction of it.
An enterprise deploys a data agent to calculate revenue growth for the previous quarter. It produces the wrong figure, in part because revenue isn’t a column in a database. “Revenue” changes with billing structures, refund rules, recognition timing, and the mix of products sold. A data agent cannot interpret these nuances.
The complexity increases when the fiscal quarter closes on the 28th rather than the 30th or 31st. It further expands when several tables in the warehouse include the word “revenue” in their names and when the one place that once clarified the correct definition, a semantic layer, hasn’t been updated since the person who maintained it left a year and a half ago.
Each gap forces additional interpretation when outdated definitions, scattered data sources, unclear lineage, and no agreed-upon source of truth are available to the AI. Each layer of interpretation demands more computation. At $6 per hour per chip, every wasted GPU hour adds to the AI infrastructure debt, fast.
Most enterprise operational systems and the data within them were designed for reporting, not AI inference. The debt has always been present. The GPU supply constraints have made it impossible to ignore.
See also: What Are Neoclouds and Why Does AI Need Them?
Why the brute-force era is over
Compute is no longer cheap, so throwing larger clusters and longer training runs at problems is a thing of the past. The years of abundant compute masked the data-efficiency gap in most enterprise AI initiatives. The companies with a strong data foundation now hold a structural cost advantage that compounds as GPU prices rise and rationing tightens.
A 1,000-GPU cluster now requires tens of millions of dollars in annual commitment to get Microsoft’s attention. General Catalyst’s Hemant Taneja, one of the most active AI investors in venture capital, surveyed portfolio founders in April 2026 about compute access because the shortage had become one of the top operational constraints (and costs) across his firms.
The standard for linking AI agents to enterprise tools and data, Model Context Protocol (MCP), does not fix the data layer. Gartner research presented at its 2026 Data & Analytics Summit flagged the same risk, predicting that most agentic analytics projects relying solely on MCP will fail by 2028 without a consistent semantic layer underneath.
In January 2026, OpenAI published an in-depth account of the internal data agent it built for its own employees, some 4,000 of them. Under the section heading “Context is everything,” it detailed the architecture required six distinct layers of context: table usage (metadata and query patterns), curated annotations, code-level definitions, institutional knowledge, persistent memory of past corrections, and run-time context. Even for one of the best-resourced AI organizations in the world, all that was essential to make a data agent reliable.
The case for data discipline
Although it remains underbuilt in most organizations, the case for data discipline is well-defined,
The data context layer concept that has crystallized across a16z, OpenAI, Palantir, and the broader practitioner community is of a governed, versioned, machine-readable repository of business definitions, source hierarchies, and semantic relationships that agents can query rather than reconstruct from scratch. Palantir has been building versions of this for years. The a16z thesis frames it as the next required architectural layer. Many refer to it as the “data product.”
A data product is a managed unit of data treated like a product rather than a byproduct of an operational system, bound by a contract that guarantees semantics, lineage, and quality signals. When an agent queries a data product rather than a raw warehouse, it doesn’t need to reconstruct context per query. The six layers OpenAI built by hand are already present.
So-called “data gravity platforms” like Databricks and Snowflake are building context surfaces, but these are compute-locked since the data product exists within the platform and is consumed through it. For enterprises whose data already spans multiple analytics and operations engines, and a separate lakehouse for machine learning workloads, a compute-locked context layer means unproductive data and disconnected intelligence.
A better approach is to place the data products and orchestrate them above the compute, so that the same contract reaches consumers regardless of which engine holds the underlying data. This is the engine-agnostic architecture in which governance, lineage, and semantic definitions are portable across the organization’s actual data landscape rather than being captive to any single platform.
Such a configuration enables unified data activation, connecting operational systems in real time. AI agents act on current positions rather than on last month’s export, and governance frameworks ensure models work with contextualized, trusted data rather than burning extra GPU cycles to compensate for ambiguity.
When GPU access is rationed, and Blackwell chips cost what they cost, data efficiency stops being best practice and becomes a competitive requirement. The enterprises best positioned for the next capacity crunch, GPUs, memory bandwidth, or whatever the next constrained resource turns out to be, are the ones that have already done the foundational work to need less of it.
