On-Ramp to EDA: Basics and Benefits

Many real-time applications work with streaming events data. That data must be ingested, processed, and analyzed in a timely manner to trigger next steps and derive insights that lead to actions. A suitable infrastructure is needed to accomplish this, with many applications either moving to or being developed for the cloud. Specifically, what is needed is an event-driven architecture (EDA).

EDA is a software development method for building applications that asynchronously communicate or integrate with other applications and systems via events. It is a way to develop applications to maximize the benefits of microservices.

Why is EDA needed? Events are used as a way of establishing communication between applications, as well as with systems. An event can be any occurrence or change in state that is identified by the application. Many applications make use of events to prompt actions.

Take a common situation, such as an online shopper adding an item to their shopping cart. An application on the site detects this and sends a message. That message can be intended for multiple uses. For example, it might be consumed by an inventory application to check for availability. It could be used by a logistics application to calculate shipping costs and provide an estimated delivery date. The same information might be used in a recommendation engine to present the shopper with some other items they might want to consider buying before checking out. And a financial application might take the information and instantly present the shopper with a discount offer if they sign up for that retailer’s credit card.

See also: EDA: Bringing Legacy Systems From Batch to Real-Time

New environments for new applications

In the past, all of these functions and applications would likely all run on a common platform managed by the retailer. In such a situation, the various applications would be tightly integrated, and the communications between the elements would be point-to-point within the aggregate system.

Today, there is a move away from such an infrastructure. Instead of tightly-integrated monolithic applications, the move is to loosely-coupled cloud-native applications. Such applications are based on containers and microservices, and they are highly distributed. That makes the exchange, passing, sharing of events more challenging.

The secret to making such distributed computing work is good solid communication between the components. Commonly used synchronous architectures cannot handle the performance and scale requirements of distributed systems. Applications based on such an architecture face challenges in a distributed environment, including communication latency, system degradation, unpredictable failures, and more.

Moving to EDA helps overcome these challenges. EDA requires minimal coupling between the services, while they can still communicate with each other. This makes EDA better suited for modern, distributed applications.

EDA basics

And event-driven architecture consists of event producers and event consumers. An event producer detects or senses an event and represents the event as a message. It does not know the consumer of the event or the outcome of an event.

After an event has been detected, it is transmitted from the event producer to the event consumers through event channels. An event processing platform processes the event asynchronously and informs event consumers when an event has occurred.

The event processing platform will execute the correct response to an event and send the activity downstream to the right consumers. This downstream activity is where the outcome of an event is seen. The consumer can use that information in different ways. For example, if the event is someone placing an item in a shopping cart, an inventory application might simply record that there is one less of that item. At the same time, an application performing a different function could take some action. As noted above, a recommendation engine might use that purchase knowledge, combine it with the customer’s past shopping experience, and present in real time several other products they might be interested in buying before checking out.

How do you get started with EDA? Apache Kafka is an open-source distributed data streaming platform that is widely used for event processing. It can handle publishing, subscribing to, storing, and processing event streams in real time. Apache Kafka supports a range of use cases where high throughput and scalability are vital, and by minimizing the need for point-to-point integrations for data sharing in certain applications, it can reduce latency to milliseconds. 

Businesses moving the EDA for the first time or scaling up grand efforts will find that, like most open-source solutions, they will need to develop (or bring in) the expertise to use EDA properly and efficiently. They also will need to add on and manage capabilities to enhance reliability and security. As is the case with many open-source solutions, companies can turn to technology partners who offer their version of the software with the needed enterprise capabilities. An example of this is Red Hat AMQ Streams.