Streaming Analytics Puts Data on the Move

Move over, static data. There’s a new capability in town, and it’s changing the way we manage insights across our enterprises. That capability, streaming data analytics, promises to move enterprises into the real-time realm.

That’s because streaming analytics reacts to events in real time. While conventional analytics provides a historical look at data and trends, streaming analytics addresses what’s happening at the moment, or provides prescriptive to address events that have yet to unfold. In a recent webcast, Tony Baer, analyst with Ovum, observes that streaming analytics – or “analyzing and making actionable decisions on data while it is in motion” – opens up a new way of thinking for many enterprises.

“You have a beacon in your store and you’re picking up a customer in aisle five,” Baer illustrates. “With streaming analytics, you can make a decision in terms of directing the customer to aisle seven to take a look at a product that he or she might be interested in.”

See also: 7 ways your business can benefit from streaming analytics

Another key difference between conventional analytics and streaming analytics is that streaming analytics has a very brief half-life, Baer adds. “This is data which is valuable in the moment but would have relatively limited value, if any at all, say an hour later.”

Not that streaming analytics is going to replace conventional analytics anytime soon — there is a place in enterprises for both. “Streaming and conventional analytics really are complementary in that one provide the context for the other,” he says. “They really are most effective when they’re used together.”

Streaming data also differs in that it is not locked up in a database until requested – representing a push versus a pull model. “There’s no there, there,” says Mark Madsen, president of Third Nature, in another webcast in the series. “There’s no place to go and retrieve your data. You have data that’s flowing across the network, and you have to tap into that. It comes to you. You don’t go to it.”

Not All Streaming Data is Created Equal

Since streaming data represents incoming streams, enterprises may not have as much control over the scale of this data as they do with conventional information. “If you have any sort of spiky workloads, which you’ll see a lot in streaming data where, suddenly, there’s 10 times as much data flowing, you then need 10 times as many resources to process that data,” says Madsen. “You have to think about how you’re going to capture and persist that data regardless of how spiky and falling it is. That creates a lot of changes to the architecture one uses to collect and persist that data.”

At the same time, not all streaming data is created equal. “You only need to know when something weird is happening — there’s an anomaly, there’s a deviation, there’s something that causes an alert,” says Madsen. That requires streaming data be managed within a long-term persistent store, to enable people and systems to quickly look at both historical and real-time patterns.

Streaming data is increasingly valuable because it enables people “to see things, get alerts, look at it, maybe go back into the past,” Madsen adds.

Much of this capability won’t necessarily be viewed by human operators but will be part of embedded systems. Ultimately, it is the metrics that matter, versus the actual data flowing underneath, Madsen explains. “Secondary metrics form what you really monitor, not the primary data that sits below it.” The traditional BI approach “tends to miss the fact that, often, you are watching an embedded system, you are not watching the data that flows in the embedded system,” he says.

For many executives and managers, the thought of adding streaming analytics and data to their infrastructures may seem daunting. Mike Lock, vice president and principal analyst for Aberdeen Group, points out that many analytics solutions will be built into vendors’ product releases. “It’s not just a one-off implementation of some dashboard or reporting capabilities. It’s much more likely to be incorporated into the product roadmap.”