Data Artisans Aims for Stream Processing at Scale

Thanks to the Internet of Things, mobile devices, cloud-based apps, geo information, social media activity, data is everywhere. It’s driving the way businesses generate revenue, service customers and develop new services.

But, of course, not all data should be treated the same. Some can be stored, analyzed and acted upon later. Some data should be processed faster and analyzed in near real-time. And lastly, some data can’t wait.  Streaming unbounded data allows companies can act on it in real-time. (Unbounded does not have a beginning and end.)

Stream processing at scale

This is where Apache Flink comes in. The open source platform is built to handle stream processing at very large scale, unify data processing from batches to event-driven microservices. It also boasts a large and active community.

Data Artisans knows a thing or two about Apache Flink. Its CEO, Kostas Tzoumas, was one of the creators of Flink, which is used by companies such as ING, Aliaba, Netflix and Uber for everything from detecting fraud in real-time to Intelligent alerting and forecasting to increasing ecommerce conversion rates to processing more than 1 trillion daily events.

[ Related: How Under Armour manages IoT streaming data ] 

Data Artisans’ dA Platform, which includes Apache Flink, is designed to drive real-time data use in the enterprise, helping them create and manage applications that rely on real-time data. Today the company released dA Platform 2, the newest version of it stream processing platform. The company says “platformizing stream processing at scale” to bring real-time data applications to the enterprise.

The version features Application Manager, said Tzoumas, and is designed to help companies to provide real-time data applications through a  centrally managed platform. dA Platform 2 dramatically reduces the manpower, cost, and effort required to provide a reliable and high-impact stream processing platform across an organization.

Streaming on the rise

Apache Flink processes data in real-time and can be applied to unbounded datasets, making it effective for streaming data applications, Tzoumas said.  In today’s DevOps wrold, he said, dA Platform 2, lets you go idea to production application faster. Enterprise can ensure that developer resources are well-spent (i.e. on building applications, not managing infrastructure.

Example of a Flink application for fraud detection.

The streaming analytics market is predicted to grow from $3.08 Billion in 2016 to $13.70 Billion by 2021 (a compound annual growth rate of 34.8 percent), according to B2B research MarketsandMarkets.

“The ability to react to data in real-time is mission-critical for today’s enterprises, and Flink is being rapidly adopted for large-scale stateful applications that can process data instantly,” Tzoumas said. “dA Platform 2 makes it possible for enterprises to run streaming applications in production across their organization quickly and efficiently.” The company will offer a preview on the new product at Flink Forward Berlin, which starts tomorrow.

[ Related: 7 Ways Your Business Can Benefit From Streaming Analytics ]

Application Manager, which includes Apache Flink, is designed to manage the lifecycle of Flink applications through development cycles, allow for instant deployment, provide application code updates, ensure version control for stateful applications. The new version of dA Platform is designed to meet the process and operational challenges of managing hundreds of Flink applications.

Data Artisans says dA Platform 2 will be generally available in the first quarter of 2018. Data Artisans is based in Berlin, Germany, and has received venture funding from Intel Capital, b-to-v Partners and Tengelmann Ventures.