The AresDB analytics engine leverages graphics processing units (GPUs) for growth at scale.
Uber has rolled out a new analytics engine, AresDB. This open source, real-time engine leverages graphics processing units (GPUs) for growth at scale. It’s designed to help unify and simplify Uber’s real-time analytics database solutions.
Column-based Storage
AresDB uses live and archive to store data in a columnar format in the cloud. A separate value stores the validity of each value, which is represented by one bit.
Real-time Ingestion with Upsert Support
Clients use the ingestion API to gather data and post an upsert batch. This specialized format reduces space needs but still keeps the data randomly accessible. AresDB can identify late records (those older than the pre-set archive cut-off time) and exclude them through the redo log. It adds late records to a backfill queue and the rest to the live store.
See also: Drive.ai launches self-driving taxi pilot in Arlington
GPU Powered Query Processing
Uber-created AQL language runs queries against AresDB. It uses JSON, YAML and Go objects. AresDB oversees multiple GPU devices with device management that models GPU resources in two dimensions, GPU threads, and device memory. The database helps users determine the number of resources necessary to execute a query. As long as a GPU device can provide all the required resources, AresDB can run one or several queries per device.
In the future, Uber plans to add new features including:
- A distributed design of AresDB for improved scalability and reduced operational costs
- Developer support and tools
- Query engine optimization
- An expanded feature set
