IBM Expands Spark Support With SystemML

IBM’s SystemML, originally created for its BigInsights data analytics platform, has been accepted as an Adobe Incubator project as part of the company’s commitment to Apache Spark. In June IBM announced they would donate SystemML, a machine learning technology used in predictive analytics models, as an open source method of building intelligent apps. Since then the project has generated over 90 contributions to Apache Spark.

Per the Apache SystemML web site, the project would provide “declarative large-scale machine learning (ML) that aims at flexible specification of ML algorithms and automatic generation of hybrid runtime plans ranging from single node, in-memory computations, to distributed computations on Apache Hadoop and Apache Spark.”

The open source version allows greater flexibility and scalability by enabling algorithms to be transferred to production environments without rewriting. This gives everything from large server clusters to single laptops the ability to scale data analysis, according to a Nov 30 blog post.

“This allows for domain –or industry –specific machine learning, providing developers what they need from a base code to customize applications,” Rob Thomas, vice president of development for IBM Analytics, noted in a statement.

Developers plan to integrate SystemML with Apache Spark. The platform can also be run via Python, Java and Scala.

Related stories:
Whats’ Behind the Attraction to Apache Spark
IBM Bets Big on Spark

Our most-read content:

Frontiers in Artificial Intelligence for the IoT: White Paper
Five Big Data Trends: Emerging Technologies
Why Edge Computing Is Crucial for the IoT
No Recalls: A Smart Testing Approach for IoT Devices

Liked this article? Share it with your colleagues!