The agentic era has continually challenged some of the most enduring convictions in data infrastructure design. For years, systems design has encoded a set of fundamental trade-offs, with scale and speed often positioned as competing forces. Now add affordability and efficiency to the equation, and the entire framework starts to feel unstable, as if the constraints that once balanced system design can no longer comfortably coexist.
Predictable Workloads or Agentic Complexity?
Teams still treat scale, speed, and cost as fixed trade-offs. Agent-driven workloads crush that model fast. Concurrency spikes, query paths branch, and old shortcuts fail under pressure. What looked balanced in static systems becomes a liability in live traffic. These presumptions held true in the past due to predictable workloads and low concurrency – more data meant longer processing times and more processing meant higher costs. AI-driven systems break that logic.
The real challenge is no longer data volume or cost alone but the explosion of concurrent, multi-step queries triggered by a single interaction. One request can expand into many analytical operations, as the system iteratively explores different subsets of data in real time, driving compute load and cost unpredictably. At that point, relying on aggressive reduction, sampling, or pre-aggregation as the primary safeguard for performance and affordability stops being a strategy and starts being a crutch.
See also: Why AI Systems Are Only as Good as the Data Being Fed into Them
Don’t Blame Data Volume for Performance Problems That are Really Architectural Problems
Modern analytical systems are questioning the idea that data volume is the primary determinant of performance. Infrastructure designed around human-led workflows leaned toward data reduction strategies through sampling, aggregation, or summarization to make queries faster and cheaper. Reasonably so. The goal there was optimizing for readability and periodic decision-making, as opposed to continuous reasoning – which happens to be the dominant pattern in agent-driven systems.
The lazy assumption that performance is solely determined by data volume is being forced to change by modern analytical systems. Human-led workflows favored sampling and summarization because they optimized for periodic reporting. Agent workflows run on continuous reasoning, where systems must resolve relationships on demand instead of serving static summaries.
Aggressive data reduction, in this context, can produce the opposite of its intended effect. While sampling or heavy aggregation may reduce the apparent complexity of individual queries, they also remove the fine-grained signals that multi-step reasoning depends on. Along with the missing intermediate detail goes the ability of systems to confidently resolve relationships within data. To compensate, it explores broader portions of the dataset through additional queries to reconstruct information, thereby defeating the purpose of simplification at ingestion time to reappear at runtime as higher compute load and latency.
In this context, aggressive reduction backfires. Sampling and heavy aggregation simplify individual queries, then strip out the fine-grained signals multi-step reasoning depends on. Once that detail is gone, systems compensate by firing additional retrieval steps to rebuild context at runtime. The simplification done at ingestion returns as higher compute burn and longer latency.
Changing the economics of scaling out
The Discovery Bottleneck
Consider systems like enterprise document search across millions of files, e-commerce catalog search over massive product inventories, semantic search and recommendation engines operating over high-volume user data. Before a query can retrieve the information a user is looking for, the system must first determine which parts of the dataset are worth examining. In many architectures, this discovery step remains concentrated, meaning that while more compute improves the speed of reading data once it has been identified, it does little to accelerate the process of determining what should be read in the first place. As datasets grow, this step itself can become a significant source of latency, cost, and complexity.
Consider enterprise document search, large e-commerce catalogs, semantic retrieval and recommendation systems. Before returning an answer, the system must decide which parts of the dataset are worth scanning. In many architectures, that discovery work is still concentrated. More compute speeds data reads after selection, but it barely improves selection itself, which becomes the growing source of latency and cost.
In conventional architectures, scaling out meant adding compute nodes that each repeated much of this same work. While query throughput increased, the process of identifying relevant data was effectively duplicated across every node. The result creates a frustrating pattern, where adding more compute nodes to handle growing load simply shifts a growing share of system resources toward repeated analysis instead of actual query execution.
Conventional scale-out adds nodes that repeat the same discovery work. Throughput rises, but data identification is duplicated across the cluster. Over time, the system spends more resources re-analyzing the dataset than answering the queries users actually asked.
Discovery in Practice
An AI assistant embedded in an analytics or operational workflow, for example, may be asked to investigate a sudden drop in conversions. That single request is often broken down into multiple parallel checks, including recent traffic changes, segment-level performance, infrastructure anomalies, and historical comparisons, each requiring targeted retrieval over different parts of the dataset. When the system is slow to surface the right data subsets early in the process, these branches expand rather than converge, increasing uncertainty and extending the overall reasoning path.
Take an AI assistant asked to explain a conversion drop. That single request often fans out into parallel checks covering traffic shifts, segment performance, infrastructure anomalies, and historical baselines. Each branch needs targeted retrieval. When discovery is slow, branches multiply instead of converge, uncertainty rises, and the reasoning path gets longer and more expensive.
Separating Compute from Storage
A modern architecture changes the scaling equation by separating compute from storage. It also separates compute from compute, so query execution, ingestion, and background tasks can scale independently instead of contending for the same resources. This is not the old sharding model where every node owns a subset of data and repeats broad discovery work. Stateless compute services scale independently while shared object storage remains the source of truth. As concurrency rises, the system scales execution capacity without copying data across workers.
Resilience Under Load
The key test is failure handling under real load. Distributed plans will hit retries, stragglers, and transient network issues. In a robust cloud design, these are task-level events, not system-level failures. Work is reassigned or retried without restarting the full query path, so multi-step workflows keep moving instead of stalling.
The payoff is practical
Teams add compute for bursty, agent-driven concurrency without paying the old tax of duplicated storage-side analysis on every node. Query latency stays more predictable because planning and execution fan out across elastic workers. Scale becomes an operational control, not an architectural penalty.
Perspective matters
There are two ways to treat scale. One treats growth as a permanent tax, where every gain demands proportional new cost and complexity. The other treats scale as something systems absorb through better architecture. The difference? Simple: growth either amplifies bottlenecks or distributes work.
When you think of scale as a moving target, what once seemed impossibly large becomes routine, making it possible to engage with larger challenges. It’s a cycle of pushing boundaries, normalizing them and moving them again. The objective is not to continually constrain data and workloads, but to build systems capable of accommodating them, adapting to them and expanding alongside them.
When scale is treated as a moving target, yesterday’s edge case becomes today’s baseline. The goal is not to keep shrinking data and limiting workloads to protect fragile systems. The goal is to build systems that keep performance predictable as data, concurrency, and complexity grow at the same time.
