As equipment vendors and interconnection platforms integrate Ultra Ethernet capabilities, enterprises will gain new freedom to design their AI environments around proximity, compliance, and efficiency rather than geography.
AI was a technology layer, but now it’s the backbone of enterprise strategy. From financial services to healthcare, organizations are weaving AI into decision-making, product design, and customer engagement. Yet as dependence deepens, so does the sensitivity of the data that fuels it. And the more embedded the models become, the more private the infrastructure must be. This is leading to an interest in what some call private AI that is empowered by Ultra Ethernet interconnected infrastructures.
Why the need for this? Across regulated industries, concerns around data sovereignty, intellectual property, and model governance are pushing organizations to bring AI in-house, either by building their own clusters or deploying within tightly controlled colocation environments. McKinsey projects that by 2030, up to 40% of AI workloads in Europe and the US will run on private infrastructure – not out of some nostalgia for the days of on-premises systems, but out of necessity. According to a survey by data center specialist Vespertec, a quarter of companies have already made that move, and another 70% are actively considering it. The goal? To protect sensitive data, avoid vendor lock-in, and ensure compliance amid an uncertain regulatory future.
But private AI comes with its own constraints. Training large language models requires dense GPU clusters, abundant power, and low-latency networking – conditions that only a handful of hyperscale environments have historically met. And therein lies the problem. Scaling private infrastructure means navigating one of the defining challenges of the AI era: the data center squeeze.
Demand for high-density space has far outstripped supply, power availability is becoming a gating factor, and the economics of AI-scale compute have grown increasingly steep. Yet amid these pressures, a new technological breakthrough is emerging that could redefine what “private” AI infrastructure looks like. A next-generation networking standard known as Ultra Ethernet promises to extend high-performance, low-latency connectivity beyond the walls of a single data hall, unlocking the possibility of training AI models across multiple facilities within a city, and in doing so, reshaping the physical limits of how AI is built.
The Data Center Squeeze
The AI boom has put immense strain on global infrastructure. As organizations rush to secure capacity for high-performance computing, the most coveted resources have been power and space. Traditional enterprise data centers were never designed for AI-scale workloads, with power density per rack now exceeding 50 kW in some configurations – at least ten times the requirements of legacy applications.
For many companies, in-house facilities simply can’t deliver the cooling or electrical capacity needed to sustain large-scale AI training. Even where upgrades are possible, regional power availability often isn’t. Across major markets, new builds are stalling under energy caps, permit delays, or local grid constraints. In the rare instances where new facilities do break ground, they are often pre-leased by hyperscalers before construction is complete, leaving little room for enterprise expansion.
This scarcity is reflected in the numbers. CBRE reports that European data center vacancy rates fell to an all-time low of 9% in 2025, while US primary markets hit just 2.8% at the start of the year. To keep up, hyperscalers have begun building private substations and even operating their own power plants, a level of investment that few enterprises can match. For everyone else, the market has become a game of compromise: choosing between proximity to data, availability of power, and the economics of colocation. That’s the paradox we now find ourselves in – even as demand for sovereign, in-house control grows, the physical constraints of data center infrastructure are tightening. To move forward, enterprises need more than raw compute; they need a new way to connect it.
See also: How AI Is Forcing an IT Infrastructure Rethink
Breaking the Latency Barrier
At the heart of every AI training environment lies an intricate web of GPUs, CPUs, and storage arrays, all bound together by a network protocol designed to move data at blistering speeds. Until now, that protocol has been InfiniBand. It excels at keeping latency in the microsecond range and ensuring high bandwidth between processors, but it comes with one fundamental limitation – distance. InfiniBand’s extreme sensitivity to latency means that GPUs must sit mere meters apart, typically within the same rack row or data hall. Spread those clusters across multiple campuses, and latency spikes make coordinated training impossible. As a result, AI infrastructure design has defaulted to centralization, concentrating compute power into massive, energy-hungry hubs.
However, in recent years, this model has begun to crack. InfiniBand’s tight vendor ecosystem, high cost, and geographic inflexibility make it poorly suited for the scale and resilience that private AI demands. Ethernet, on the other hand, has long been the universal language of digital infrastructure – simple, cost-effective, and globally standardized – but it was never built for the precision of AI workloads. Packet loss and congestion meant it couldn’t guarantee the deterministic performance required for training large models. That equation changed in mid-2025 with the version 1.0 release of Ultra Ethernet (UE). Developed to combine Ethernet’s openness with the low-latency performance of InfiniBand, UE introduces intelligent congestion control, precise flow management, and near-lossless transmission. It’s already enabling AI training within data centers, while the next iterations, expected sometime in 2026, will stretch those capabilities city-wide. With latency tolerances of up to one millisecond, Ultra Ethernet could transform multiple data centers throughout a metropolitan region into a single, distributed fabric, overcoming one of the biggest hurdles in scaling private AI.
Distributed AI Infrastructure
For the first time, organizations will be able to train a single model across multiple sites within a metropolitan area, effectively stitching together a city’s available rackspace into one cohesive compute environment. Instead of competing for scarce space in a single high-density facility, enterprises could aggregate capacity across several smaller colocation sites, connecting them with low-latency, high-throughput links that behave as if all the GPUs were under the same roof. When one facility reaches its power limit, workloads can extend seamlessly to another location without compromising training speed or data security.
Ultra Ethernet also builds on existing, widely deployed Ethernet hardware. That means many colocation and enterprise facilities could adopt it through software upgrades rather than costly retrofits, accelerating its rollout across global markets. As equipment vendors and interconnection platforms integrate UE capabilities, enterprises will gain new freedom to design their AI environments around proximity, compliance, and efficiency rather than geography. This creates the conditions for a new kind of ecosystem where distributed infrastructure can be interconnected through neutral, high-performance AI Exchanges (AI-IXs). These exchanges will combine Ultra Ethernet–enabled connectivity with optimized AI routing and direct links between data centers and cloud providers. By enabling distributed AI training and inference on shared, carrier-neutral platforms, AI-IXs will blur the traditional boundaries between compute, connectivity, and colocation, putting private, scalable AI firmly within reach.
