Getting Fast Results from Slow Data

fast data

Fast analytics over fast data. Makes perfect sense, right? How about fast analytics over slow data? Doesn’t make so much sense, does it? But it does and it can be just as valuable. Here, RT Insights founder and Executive Editor Les Yeamans shares a new white paper written by Dr. Dale Skeen, cofounder and CTO of operation intelligence company Vitria Technology, Inc., that explains it all.

We’re bombarded with fast data every day. News, financial tickers and sensor data are three examples. Picture a waterfall. That’s fast data, and fast analytics helps us sort through and use that data in the most efficient way possible. Now picture a lawn sprinkler system which only releases water at preset intervals. That’s a system with a slower data cycle operating twice a day. Just as a lawn sprinkler system is crucial to keeping your favorite golf course green and pristine, slow data is valuable, too. In today’s Internet of Things (IoT), there’s a lot of this type of data being generated.

Let’s look at one example. Many utility companies now use smart meters which monitor a home’s electrical use and report it back to the company. These readings happen once an hour. On the surface, this seems fine and more than adequate to gather the usage data needed for billing purposes. Then a heat wave strikes and people start turning on air conditioners and fans. Naturally, this will cause a spike in usage – sometimes large enough to cause brownouts (or worse) because the utility company isn’t notified of the spike right away.

This is where fast analytics over slow data comes in. While a utility company may have those meters set to report usage data once an hour, every individual meter is set to report its usage at a different time during that hour. With fast analytics, you can leverage that individual data with amazing results as shown in the white paper for a 15 million smart meter grid:

  • Within five minutes into the data cycle, we can detect a variance from actual demand based on 1.25 million new readings
  • Within 10 minutes into the data cycle, we can predict a shortage of capacity based on 2.5 million new readings
  • Within 15 minutes into the data cycle, we can predict an energy shortfall with over 99 percent confidence level based on 3.75 million new readings and, given this high confidence level, start to spin up a generator to meet the surge in demand
  • Within 45 minutes we can bring a new gas turbine online just in time to avoid brownouts or worse

Now we have serious value! By using fast analytics over slow data, problems and anomalies can be detected early and problems can be predicted before they happen so that a plan can be put into motion to prevent them.

Les Yeamans

About Les Yeamans

Les Yeamans is founder and Executive Editor of RTInsights and CDInsights. He is a business entrepreneur with over 25 years of experience developing and managing successful companies in the enterprise software, financial services, strategic consulting and Internet publishing markets. Before founding RTInsights, Les founded and led, an Internet portal company specializing in the application of critical enterprise technologies including BPM, event-driven architectures, and event processing. When was acquired by TechTarget, Les became Associate Publisher, managing a group of websites. Previously, Les had founded a new enterprise software business called ezBridge which provided fault-tolerant, guaranteed delivery transaction messaging on 10 different hardware platforms. This product was licensed to IBM as the initial code base for IBM MQSeries (renamed WebSphere MQ and later renamed IBM MQ) which was co-developed and co-marketed with IBM. Les was also co-founder of the Message Oriented Middleware Association (MOMA). Les has worked extensively as an analyst and consultant for end users and vendors in this growing market. Prior to ezBridge, Les raised venture capital for development and marketing of PowerBase, the industry-leading database software package for the IBM PC. He started his career consulting at Accenture, providing end-user IT solutions. Les has an MBA from the University of Michigan and a Bachelor's degree from the State University of New York at Binghamton. He is based in New Rochelle, NY.

Leave a Reply

Your email address will not be published. Required fields are marked *