Center for Continuous Intelligence
Continuous intelligence relies on platforms, architectures, and software that lets organizations use real-time and historical data to enable critical decisions and actions to be taken in milliseconds to minutes.
By integrating continuous intelligence systems into business processes using real time and historical data, organizations can respond in near real time.
Real-time analytics have increasingly become a competitive advantage for the modern enterprise.
Experience Continuous Intelligence (Webinar)
IBM Streams is available through Cloud Pak for Data. Learn more
Subscribe to the CI newsletter
What Is Continuous Intelligence?
Continuous intelligence (CI) gives organizations the information needed to react to events as they are happening. It is a form of real-time analytics used to support decisions when actions must be taken in milliseconds to minutes.
Continuous intelligence relies on platforms, architectures, and software that allows organizations to collect, organize, and analyze data to enable fast actions in response to real-time events. Unlike traditional analytics, it relies on both the analysis of historical data and real-time data from a variety of sources. That data can include email messages, clickstreams, social media, data logs, and information from sensors and Internet of Things (IoT) devices.
In most cases, CI makes use of artificial intelligence (AI) and machine learning (ML) to complement traditional analytics used to investigate these datasets. Businesses can realize significant benefits from CI when their real-time analytics capabilities are integrated into their operations. In this way, CI can prescribe actions in response to business moments and other events.
What makes CI different?
CI enables smarter business decisions using real-time data streams and advanced analytics. It differs from traditional analysis in that it is always on for situational awareness, prescribes actions, and allows businesses to be proactive.
In contrast, conventional operational decisions are made by doing real-time calculations on historical data or data captured at one moment in time. A simple example that illustrates the difference is an app that calculates the distance users have walked each month using global positioning system (GPS) location data. The traditional approach would make one calculation using all the data transmitted from the user’s mobile device and stored over the past month. In contrast, CI using stream processing performs continuous analytics to keep running totals that are updated moment by moment as the GPS data is refreshed.
In this way, CI augments the traditional analytics approach by enabling continuous analysis to be modified over time. For example, the streaming data from a user’s phone can be analyzed using machine learning techniques to determine the most popular walking routes or detect deviations from a user’s standard walking patterns.
Another powerful example that highlights the differentiation of CI from tradition real-time analysis is machine maintenance. A traditional approach is to either wait until something breaks and fix the problem, do preventive maintenance by replacing parts on a pre-determined schedule, or conduct periodic manual inspections.
CI enables predictive maintenance. Continuous sensor-based monitoring could be used to identify leading problem indicators and implement a just-in-time, condition-based replacement of parts. The instantaneous information from sensors also can be analyzed in real time to optimize a machine’s performance.
Why is CI going mainstream now?
Organizations need to identify risks and opportunities in high-velocity data—opportunities that often can be detected and acted on only at a moment’s notice. Flows of data from streaming sources (such as market data, the IoT, mobile devices, sensors, clickstreams, and even transactions) remain largely unnavigated. Such data must be unlocked to optimize decision making.
To innovate, organizations must quickly make sense of all this data—from persistent structured data to streaming data from sources such as smart sensors and IoT devices—decisively, consistently, and in real time. Understanding the context of data is key to improving customer relationships, enhancing operational efficiencies, reducing risk, and uncovering new opportunities. CI is the instrument that derives that context.
Leading-edge applications such as mobile device navigation systems, equity trading systems, e-commerce site referral engines, and others have made use of CI for years. For example, mapping apps combine real-time location data from every traveler’s mobile phone to determine the concentration of cars on the roads and how fast they are moving. Some add in updates from Departments of Transportation from across the country to pinpoint accidents. And in the latest iteration, some mapping apps add in crowdsourced information about delays, accidents, and debris in the road provided by drivers and passengers. The information is synthesized to determine estimated time of arrival at a destination and to suggest alternate routes, all in real time.
Now CI is being more widely adopted across many industries and for many applications. The reason: There is a perfect storm of data sources and tools. Businesses have huge amounts of streaming data from IoT devices, smart sensors, and other sources that are ripe for analysis and inclusion in business processes. And the technologies (hardware, software, services, and systems) to make use of that data are now available.
The combination of lots of streaming data and solutions to derive actionable intelligence from that data means CI can deliver significant benefits to businesses of all types and sizes. A financial institution could use CI for real-time fraud prevention by detecting malicious transactions and stopping them before they are executed. An online retailer could use CI to provide an enhanced customer experience and improved service when a customer contacts a call center or moves through the product selection and purchasing process online. First responders and agencies dealing with natural disasters could use CI to deploy resources more efficiently as a dynamic situation unfolds.
CI’s impact is expected to accelerate its adoption. By 2022, more than half of major new business systems will incorporate continuous intelligence that uses real-time context data to improve decisions, according to Gartner analyst Roy Schulte.
Key elements to CI success
Several factors make CI more accessible to the masses today. They include:
Hardware: Businesses today have access to the high-performance computing capabilities needed for CI. The price of CPUs, GPUs, and high-performance memory and storage continues to drop, allowing businesses to install systems with processing power that rivals that of supercomputers of five years ago.
Businesses also have the option of using high-performance CPU, GPU, storage, and interconnect instances from the major cloud service providers. Getting a CI initiative started does not require a significant upfront investment with this approach. Additionally, cloud services allow businesses to scale up efforts over time quickly.
Analysis, AI, and ML software: A second factor assisting CI in becoming mainstream is the availability of new analysis algorithms to make sense of streaming data. Today, businesses have access to relatively easy-to-use machine learning, artificial intelligence, stream analytics, and time-series DBMS software.
Cloud and middleware: The growing use of cloud-native applications, microservice architectures, and hybrid cloud makes it easier for businesses to develop, deploy, and run CI across the enterprise. And modern middleware gives businesses the ability to move, host, and access CI applications and data on-premises or in the cloud, taking performance and cost factors into account.
Data platforms and architectures for CI
The heart of any CI effort is the streaming and historical data to be analyzed. Managing the data, preparing the data for analysis, ensuring access to the data, and safeguarding the data can be formidable tasks.
The right data platform and architecture would simplify and unify how businesses collect, organize, and analyze data to accelerate the value of real-time analytics and AI.
Additionally, what’s needed is a hyper-converged architecture that combines storage, computing, and networking software into a single system. Such a solution can simplify and unify how a business manages, governs, and analyzes data. The right solution allows businesses to provision and deploy data services flexibly and rapidly.
Hyper-converged technology makes it possible for businesses to scale and evolve their infrastructures simply and economically as application loads change. An ideal solution would use a flexible microservices software architecture and acceleration hardware designed for the compute needs of stream processing and AI.
The CI paradigm shift
Making use of these technologies for CI gives businesses a way to radically change the way they make decisions.
Traditional analytics and business intelligence (BI) are descriptive, providing information on what happened, and diagnostic, helping to explain how something happened. For example, traditional analysis of merchandise sales would note that a clothing line had a sudden spurt in popularity, and it was due to a social media influencer’s endorsement of the line.
More sophisticated uses of analytics and BI also are often predictive. For example, an analysis might use machine learning to help infer what is likely to happen next. An example would be a machine maintenance application where analysis of historical data identifies that a machine of this model, this age, and operating at this temperature, is likely to have a specific part fail after these many hours of service.
Decisions made using these methods are reactive and are based on historical data. In the case where a machine part is identified as having a high likelihood of failing, that part can be replaced before it fails and disrupts operations. But the problem with this approach is that it does not use situational information to make smarter decisions. A part might be replaced to be on the safe side sooner than it needed to be. That practice results in spending money on new parts while the old are still functioning.
In contrast, CI would use real-time information about the part to find that its performance is degrading at a slower-than-normal rate, so may remain in place for a longer time. This approach saves money by extending the time a part is used, while also ensuring the part will not fail. Such applications get to the heart of CI’s strength.
CI use streaming data to understand what is happening and is embedded within business operations. In this way, CI can derive real-time context data from event streams. Those streams can include:
- Transaction records such as customer purchases, bank transactions, and more
- Interactions including clickstreams, emails, customer call logs, and social media posts
- Related data such as IT network logs, market data, and news feeds
- Physical events data captured by sensors and IoT devices including machine logs, geolocation, telemetry, and SCADA data
CI based on such event streams enables proactive responses (like supply chain visibility, facility monitoring, etc.) and reactive responses (such as providing a customer with a next-best offer or checking transactions for fraud). It takes decision-making to a new level. In the past, predictive analytics might be used to determine what will happen next. A machine part will fail, for example. CI can complement this predictive capability using prescriptive analytics or rules to determine what to do next. As such, CI can be used for decision support and decision automation.
CI used for decision support helps businesses and people make decisions on how to respond to events that may be rapidly changing. The derived intelligence complements the actions a person would take. So rather than making a decision based on a gut feeling (a change in market conditions, for example), CI would provide the information to back up the decision. General application areas for CI in business include risk assessment, target marketing, sales acceleration, revenue growth, finding opportunities for funding business growth, and increasing operational efficiencies.
CI for decision automation takes things to a higher level. The derived information is used to take actions without a person involved. CI for decision automation has many benefits. It offloads work from people; is a faster, less expensive way to operate than relying on human decisions; ensures more consistent decisions; and can guarantee better compliance by following predetermined policies.
Bringing it all together
CI helps businesses make decisions while events are happening. It brings meaning to fast-moving data streams and helps organizations in a wide variety of industries.
Today, data streams that business have available and should make use of for CI can originate from many sources, including:
- IoT devices and sensors
- Text files, spreadsheets, images, video, and audio recordings
- Email, chat, and instant messaging
- Web traffic, blogs, and social networking sites
- Financial transactions, customer service records, telephone usage records, and system and application logs
- Satellite data, GPS data, smart devices, sensors, network traffic, and messages
Solutions to support CI must include appropriate hardware and real-time analytics and artificial intelligence software. The architecture to support CI must be capable of storing, managing, and safeguarding historical and streaming data. Solutions must be deployable on-premises and in the cloud, and easily moved to the platform that offers the best performance and cost as conditions change.
With the right platform and architecture, a CI solution will be able to:
- Continuously analyze data in motion across multiple sources to deliver actionable insight.
- Connect to any data stream to make predictions and discoveries as data arrives to enhance and improve analytic models and cognitive systems.
- Deploy a complete set of streaming analytics—such as natural language processing, geospatial, predictive, and more—to satisfy unique, industry-specific requirements, and use cases.
- Speed time to value with open-source technologies using APIs and visualize data easily with drag-and-drop development tools that help support faster time to deployment and effective production management.