The Challenge of Full-Stack Observability

Full-stack observability promises to provide better visibility into the complete – and complex – enterprise IT environment. With that comes a sharper view into the customer experience. Yet, IT professionals say that view is a bit too fragmented today.

That’s a key takeaway from research conducted by IT advisory firm SiliconANGLE Media and its survey partner Enterprise Technology Research (ETR). The two firms recently published Cutting Through the Noise of Full-Stack Observability. The report analyzed the needs and goals of 88 enterprises. It looked in depth at how well the software sector is addressing those requirements.

It was what you might call a mixed bag with software vendors tackling new responsibilities and users still discovering what they want observability to accomplish.

Full-stack complexity

The report highlighted why full-stack observability is a challenge.

Consider the very nature of today’s full-stack architecture. It’s becoming more complex almost by the day. Modern applications are built with a mashup of elements, from chunks of code to remote, cloud-based services. The sheer number of enterprise-wide applications is commonly estimated at well over 100 on average. And, one-off departmental apps make a portfolio a challenge to manage. Add in the complexity of multiple client types, remote access, and distributed networks, and you have complexity.

“We’re talking about the compute, storage, networking, containers, the database and other middleware components, the pipeline of data; and the client-side code: the HTML, the CSS, everything down to the mobile device,” wrote SiliconANGLE co-CEO Dave Vellante.

Vellante added, “Every vendor wants a piece of the action in this market, including companies that have provided traditional monitoring, log analytics, application performance management and similar services. These companies are joined by a slew of new entrants claiming end-to-end visibility across the so-called ‘modern tech stack’.”

From point products to full-stack observability

So, the environment itself is complex. Then, the solutions end up being complicated as well, with multiple software providers each offering to see and manage it all. One problem is that a vendor may be good at one aspect, such as log analytics, but not so good at another. The research concluded that the most likely path to success is for multiple vendors to bundle their offerings.

However, it’s not just the environment and vendor offerings that are complex. Even the type of manager who buys observability tools varies greatly.

Who can speak for an organization’s observability needs? The research addressed eight different roles and groups. Those involved in security operations, DevOps, infrastructure, and network operations were most likely to use observability.

However, Erik Bradley, chief engagement strategist at ETR, said he was surprised that application operations and site reliability engineers (SRE) weren’t more involved with observability. “This isn’t just one domain. It’s touching every aspect of the enterprise, and that’s including the internal IT teams.” He noted that SRE’s and application teams are the people who are likely to be charged with ensuring good customer experience.

Price points

Those IT pros interviewed as part of the research said that pricing for observability tools is an issue for user organizations. “It really is all over the map from a vendor perspective, and also from an end-user perspective. Of all the interviews and panels that I host, pricing’s a real concern. It is always, but in this particular space, it’s a real concern,” said Bailey.

One aspect of the pricing issue is that of the software pricing model being used. Respondents were given the choice among models based on host-based pricing (determined by the infrastructure being monitored), user-based pricing (number of seats), and a data ingestion/usage model. Respondents preferred the predictability of a host-based model for budgeting purposes, while others liked the flexibility that the ingestion model offers.

The research also focused on how organizations are using observability. The results varied significantly, depending upon which vendors respondents were using.

In general, application performance management was the most popular function to be targeted with observability, followed by database monitoring.

Observing the customer experience

What surprised Vellante and Bradley was the relatively low use of observability tools for managing the digital experience of end users and company customers. This was “despite all the chatter in the market about digital transformation and the customer experience…so the vendor marketing doesn’t match the user behavior”, according to Vellante.

“I did expect it to be more. And not just from the vendor marketing perspective, but you and I both know at the end of the day, the whole point of this is to actually get into that 360 view of what your customer is doing,” said Bradley. However, Bradley did note that some organizations said they are using technologies other than observability tools to monitor customer experiences.

That the full-stack observability market is fragmented, as is the approach to buying and using the tools, shouldn’t come as a surprise. The observability concept, which takes a giant leap beyond traditional systems monitoring, is still in the early stages. Plus, the suppliers and users of those observabilities are dealing with dramatic shifts in corporate structures, the nature of the IT, and how applications are built and delivered. Rome wasn’t built in a day.