SHARE

IBM Launches Cloud Development Hub for Apache Spark

Data scientists can analyze big data quickly with the IBM Cloud Bluemix platform-based hub.

Written By

Jun 8, 2016

IBM has announced the creation of the first cloud-based development environment for near real-time analytics using Apache Spark. In the June 7 announcement, the company said the new hub, which they’ve named the Data Science Experience, offers 250 curated data sets, a collection of open source tools and a collaborative workspace, all designed to assist data scientists in finding and sharing meaningful insights with developers. It’s hoped that these insights will result in more rapidly developed applications.

According to IBM’s press release, the new hub builds on IBM’s $300 million investment in Apache Spark. Billed as a type of analytics OS, the Data Science Experience will extend Spark to over two million members of the R community. This will be achieved though new contributions to SparkR, SparkSQL and Apache SparkML, the company stated. This means data scientists who work in R will have faster access to increased data, which means more insights.

“With Apache Spark, we see an opportunity to significantly transform the role of the data scientist by providing access to curated data sets, open source tools and a collaborative platform to accelerate innovation,” said Bob Picciano, senior vice president of IBM Analytics.

The open and collaborative environment will enable data scientists to bring in data and other open-source resources from IBM, Jupyter Notebooks, H2O, RStudio and many others in a single and secure environment. Curating and analyzing that data will be sped up and simplified, the announcement explained.

IBM has made over 3,000 contributions to analytics related projects in the last year, including Apache Toree, EclairJS, Apache Quarks, Apache Mesos and Apache Spark sub-projects SparkSQL, SparkR, MLLib, and PySpark.

Spark has also been baked into the core of its popular Watson IoT platform as well as their commerce, analytics and cloud platforms. In addition, they have 30 other Spark-based offerings available, the company stated in their release.

Why Apache Spark Is Hot

Sue Walsh

Sue Walsh is News Writer for RTInsights, and a freelance writer and social media manager living in New York City. Her specialties include tech, security and e-commerce. You can follow her on Twitter at @girlfridaygeek.