Databricks Lowers AI Barrier to Entry with Unified Analytics

Databricks, the company comprised of the creators of Apache Spark, announced the Unified Analytics Platform at the Spark + AI Summit 2018 in San Francisco, California. The platform aims to lower the barriers for enterprises looking to integrate artificial intelligence (AI) into their business, by unifying data and AI technologies into one workflow.

Databricks found that in a commissioned research study, 95 percent of organizations say data silos are the largest obstacle for companies that want to add AI to production. It claims the Unified Analytics Platform ensures that an organization’s data and AI efforts are unified, improving workflow, data reliability, and performance at scale.

See also: How AI Makes Real-Time Analytics More Real

“To derive value from AI, enterprises are dependent on their existing data and ability to iteratively do AI on massive data sets. Today, data engineers and data scientists use numerous, disconnected tools to accomplish this, including a zoo of machine learning frameworks,” said Ali Ghodsi, co-founder and CEO at Databricks.

“Both organizational and technology silos create friction and slow down projects, becoming an impediment to the highly iterative nature of AI projects. Unified Analytics is the solution to unify data processing and AI technologies, and increase collaboration between data engineers and data scientists across the AI lifecycle.”

Databricks’ MLflow aims to simplify the machine learning workflow by allowing organizations to package their code for multiple runs, compare hundreds of experiments, leverage a multitude of hardware and software platforms, and deploy on a variety of platforms. MLflow is integrated with a number of open-source machine learning frameworks, including Apache Spark, TensorFlow, and SciKit-Learn.

“When it comes to building a web or mobile application, organizations know how to do that because we’ve built toolkits, workflows, and reference architectures.  But there is no toolkit for machine learning, which is forcing organizations to piece together point solutions and secure highly specialized skills to achieve AI,” said Matei Zaharia, co-founder and Chief Technologist at Databricks.

“MLflow is a unified toolkit for developing machine learning applications in a repeatable manner while having the flexibility to deploy reliably in production across multiple cloud environments.”