Solutions/Observability

Observability
on VeloDB

Logs, metrics, and agent traces in one engine. Search billions of rows of logs in seconds, keep months of history, and pay a fraction of what Elasticsearch or Datadog costs.

Store application logs, infrastructure metrics, and AI agent traces in one table layout, queried with SQL and full-text search

Ingest deeply nested, constantly changing JSON without ETL, flattening, or mapping explosions

Run phrase search, JSON filters, trace replay, and dashboard aggregations against the same data

Keep full retention with tiered storage: hot data on SSD, history on object storage

Collect with OpenTelemetry, Vector, or Logstash and visualize in Grafana, open source at every layer

Start Free Trial

Book a Consultation

VeloDB log search across infrastructure logs and AI agent traces

One engine for agent traces
and infrastructure logs

AI agents produce telemetry data that breaks traditional logging solutions: payloads are much larger, growing from 5 KB to 1 MB, JSON schemas can also evolve with every tool call, and traces that only make sense when replayed in order. VeloDB is designed to handle this new workload and your classic logs, metrics, and traces on the same engine.

Built for agent telemetry

The VARIANT data type stores nested, evolving trace payloads and automatically turns frequent JSON paths into columns. Query payload["tool_call"]["tool_name"] with columnar speed, no schema declared up front.

Search and analytics on the same table

Inverted indexes return keyword matches in milliseconds while columnar storage powers aggregations, and SQL joins correlate logs with metrics and business data for root cause analysis.

Cost that holds at petabyte scale

Columnar storage with ZSTD compression and a simplified inverted index cuts storage to roughly 20% of an Elasticsearch footprint. Tiered storage moves cold data to object storage and cuts unit cost by more than half.

Open at every layer

OpenTelemetry for collection, Grafana for dashboards, MySQL protocol for every SQL client, Apache Doris (Apache 2.0) underneath. Your telemetry stays in open formats you can leave with.

Trusted in production

Observability teams run on VeloDB

MiniMax runs a PB-scale log system for every business line on one architecture, replacing Grafana Loki.

Log records queried in 2s

10GB/s

Write throughput

70%

Storage cost reduction

“Loki scanned full datasets for every regex search and needed a separate deployment per cluster. On Apache Doris, inverted indexes answer a query across 1 billion log records in 2 seconds, one architecture serves every cluster, and availability holds above 99.9%.”

Platform Engineering Team, MiniMax

PB-scale logs, 99.9%+ availability

Read the full story

MiniMaxStepFunNetEaseAdvance.AIKwaiTencent MusicJD.comBaiduMeituanTrip.comMiniMaxStepFunNetEaseAdvance.AIKwaiTencent MusicJD.comBaiduMeituanTrip.com

Real-world tradeoffs

Challenges with Observability
in the Agentic Era

01·Cost

Observability bills now rival the infrastructure they monitor

Elasticsearch-based stacks need 12 to 19x more storage than columnar alternatives.

So teams sample logs, shorten retention, and lose the history they need for debugging and compliance.

Tap to flip

How VeloDB solves it

Full retention at roughly 20% of the Elasticsearch footprint

Columnar storage, a simplified inverted index, and ZSTD compression hold search performance at roughly 20% of the Elasticsearch storage footprint. Writes use 80% less CPU. Tiered storage keeps recent data on SSD and moves history to object storage, so keeping full history fits the budget.

← Flip back

02·Wide JSON

Agent traces and wide JSON break fixed schemas

Agent telemetry is deeply nested JSON with schemas that change constantly, sometimes past 10,000 distinct fields.

Flattening breaks on schema evolution, storing JSON as strings forces full scans, and Elasticsearch answers with mapping explosion.

Tap to flip

How VeloDB solves it

VARIANT stores evolving JSON and reads it as columns

VARIANT stores JSON semi-structured on write and columnar on read, extracting hot paths into real columns automatically. Doris 4.1 handles tables beyond 10,000 JSON fields with deferred shredding, opening segments 16x faster with 60x less memory on ultra-wide tables.

← Flip back

03·Access patterns

Four access patterns collide in a single incident

One investigation needs phrase search over raw text, filters on JSON payloads, ordered trace replay, and live dashboard aggregations.

Teams run a search engine, an OLAP database, and a document store side by side because no one system covers all four.

Tap to flip

How VeloDB solves it

One table layout serves all four access patterns

On AgentLogsBench, a 100-million-trace agent observability benchmark, Apache Doris leads the combined leaderboard: trace replay in 0.020s where ClickHouse takes 2.289s, incident search in 0.088s where Elasticsearch takes 1.116s, and JSON filters in 0.078s where Elasticsearch takes 0.802s.

← Flip back

04·Lock-in

Telemetry locked in closed formats and vendor query languages

Commercial observability platforms hold telemetry behind proprietary formats and query languages.

Costs climb yearly, 36% of enterprise clients now spend over $1M a year on observability, and leaving means rebuilding every dashboard, alert, and saved query.

Tap to flip

How VeloDB solves it

Open at collection, visualization, and engine

The collection layer is OpenTelemetry, with a Doris exporter contributed to the OTel community that auto-creates tables and manages retention from YAML config. The visualization layer is Grafana, connected over MySQL protocol with a dedicated Doris plugin for log discovery and trace views. The engine is open-source Apache Doris.

← Flip back

Architecture overview

VeloDB for observability

Collect once with open agents, ingest continuously, store logs, metrics, and traces in one engine, and serve search, dashboards, and alerts from the same tables.

VeloDB observability engine

Collect

OpenTelemetry

Doris exporter · logs, traces, metrics

Log Shippers

Vector · Logstash · Filebeat · iLogtail

Kafka

Buffer for high-volume pipelines

LLM Instrumentation

Langfuse · LangChain · OpenAI · Claude

Ingest

Routine Load