Uber Introduces Open Source, Real-Time Analytics Engine

Uber has rolled out a new analytics engine, AresDB. This open source, real-time engine leverages graphics processing units (GPUs) for growth at scale. It’s designed to help unify and simplify Uber’s real-time analytics database solutions.

Column-based Storage

AresDB uses live and archive to store data in a columnar format in the cloud. A separate value stores the validity of each value, which is represented by one bit.

Real-time Ingestion with Upsert Support

Clients use the ingestion API to gather data and post an upsert batch. This specialized format reduces space needs but still keeps the data randomly accessible. AresDB can identify late records (those older than the pre-set archive cut-off time) and exclude them through the redo log. It adds late records to a backfill queue and the rest to the live store.

See also: Drive.ai launches self-driving taxi pilot in Arlington

GPU Powered Query Processing

Uber-created AQL language runs queries against AresDB. It uses JSON, YAML and Go objects. AresDB oversees multiple GPU devices with device management that models GPU resources in two dimensions, GPU threads, and device memory. The database helps users determine the number of resources necessary to execute a query. As long as a GPU device can provide all the required resources, AresDB can run one or several queries per device.

In the future, Uber plans to add new features including:

• A distributed design of AresDB for improved scalability and reduced operational costs
• Developer support and tools
• Query engine optimization
• An expanded feature set