Sponsored by Vantiq
Center for Real-Time Applications Development

Testing Tools and Considerations for Real-Time Applications


Robust testing means that your Real-Time Application is more stable and reliable than ever before.

When building a Real-Time Application, any engineer would agree that testing the Application is half of the battle. Creating tests that fully cover every scenario is challenging and time-consuming.

Applications are becoming faster and easier to build in new low-code environments. As the Application creation process is revolutionized, the test creation process must be quick to follow; otherwise, the quality of Applications will begin to suffer.

Rethinking Test Inputs and Outputs

In traditional systems, Application logic is linear and synchronous, as are the tests.  A test is a sequence of steps. If X is provided as an input, then Y is expected as an immediate result. Y may then be provided as the input for the next step in the test and so on.

The traditional framework quickly breaks down for real-time asynchronous systems. As a result, the test input and output model must be reinvented. For example, a developer might have to test a model that expects event C to occur if events A and B occur within 1 second of each other. In a real-time environment, a test is not a sequence of steps to be verified one after another. A real-time test is defined by a set of input events that will be created by the system and a set of output events that are expected to occur before the test terminates.

Input events are described by the type of event and the event’s contents. Inputs may be spaced and timed to best mimic real-time data arriving into the system.

Output events are described by the type of event, the event’s contents, and the number of times this event is expected to occur. Output events should exist independently of any particular input or set of inputs. The crux of a real-time test is that at runtime, the test waits for each output to occur, agnostic of how or why it occurred.

Low-Code Test Development

Describing Input and Output events individually is effective for creating small targeted tests but can become tedious when describing an Application that ingests thousands of events per second. When designing a full suite of real-time testing tools, features that enable developers to efficiently create realistic test inputs is key to success.

Event Mocking

The more closely a test can mimic real scenarios, the more useful the test becomes. However, real data is complex, and tests that accurately reflect real-world data can be arduous to build.

Creating a tool to help developers simulate real-world data without manually creating each individual event increases productivity by an order of magnitude. Developers should be able to flexibly describe any type of events that can be produced. By leveraging known event content schemas and using random generators, ranges, and specified sets, thousands of events can be described with little to no code.

By creating realistic events on demand, Event Mocking tools are useful for manual testing and troubleshooting during development. They can be used as test inputs to avoid redundant event definitions or even for performance testing to guarantee your system works correctly under load.

This image shows an Event Generator, which provides simulation data to test an application while the simulated events are recorded. The developer can then track the data flowing through the Application using the visualization graphs to manually test that the Application is working correctly.

Capture Events and Replay

Using Event Mocking to allow users to describe a series of events is particularly useful for creating controlled test scenarios. However, it is almost impossible for a developer to create 100% authentic test data purely with an Event Mocking Tool.

If the goal is to mimic real scenarios, then the best solution is to go straight to the source. Rather than manufacturing data to imitate events, tests can actually use real events.

Mechanisms that record the events that occur within an Application over a period of time can allow users to capture and replay real events. The Captured events can be exported to a test and set as test inputs and outputs. Developers can optionally transform and tweak the captured data before exporting it. This allows users to export the same captured data into several tests to produce slightly different scenarios.

By using actual events recorded by the system, developers can rest easy knowing that their tests accurately exercise situations that their Application will experience.

A New Era of Testing

A new generation of Real-Time Event-Driven Applications are allowing developers to reinvent the Application testing process. Tests with thousands of inputs and outputs that perfectly reflect real-time data can be created in mere minutes with little to no code required. Robust testing means that your Real-Time Application is more stable and reliable than ever before.

Daphne Nhuch

About Daphne Nhuch

Daphne Nhuch is a full stack Software Engineer at VANTIQ. Since joining Vantiq, she has architected and implemented Advanced Event Broker Catalog and Event Driven Testing Tools. Daphne graduated from UC Berkley in 2019 with a major in Computer Science with a focus on Databases.

Leave a Reply