VeloDB and Apache Doris Get Rust-Native Streaming Ingestion with Apache Iggy

Real-time analytics depends on two things working together: landing events fast, and querying them fast. Slow either half, and the pipeline stops feeling real-time. Today, Apache Doris and VeloDB users get a faster path for the ingest half, built together with the team at LaserData: a Rust-native sink connector that streams events from Apache Iggy straight into Apache Doris and VeloDB Cloud.

Apache Iggy is a next-generation Rust-native data streaming platform built for modern real-time and AI-native systems. It combines ultra-low latency, predictable tail performance, and resource efficiency into a foundation for event-driven architectures, real-time analytics, edge computing, and agentic AI. It moves millions of messages per second at roughly 1 ms write latency on a Rust, io_uring, thread-per-core runtime, with no JVM and no garbage-collection pauses. The new connector reads Iggy topics, batches rows, and loads them into Doris through Stream Load. A single Rust-native pipeline streams events directly into Apache Doris with fewer moving parts, lower operational overhead, and significantly lower tail latency. One Rust seam connects a fast streaming engine to a fast database.

This integration is a joint effort. VeloDB and the Apache Doris community worked with LaserData and the Apache Iggy community to ship native connectors on both sides and to benchmark the combined stack end to end. Iggy owns the fast-in half. Apache Doris, the real-time analytics database behind VeloDB, owns the fast-out half.

A Rust-native stack means more than the implementation language. It enables predictable memory usage, no garbage collection pauses, lower infrastructure costs, and better utilization of modern multi-core hardware. As data volumes continue growing, software efficiency increasingly translates directly into infrastructure efficiency.

Kranti Parisa, Founder & CEO, LaserData Inc & Apache Iggy PPMC

Apache Iggy is developed by the Apache community with significant contributions from LaserData, the company building next-generation data infrastructure for real-time and AI-native systems.

A single Rust seam

A typical streaming path into Doris chains several systems in sequence:

Producers -> Kafka (JVM broker) -> Kafka Connect -> Stream processor -> Apache Doris
                        GC pauses                    extra cluster          serialization hop

Each additional hop in the tool chain causes additional latency. A JVM broker adds tail latency whenever garbage collection pauses, so fresh events queue behind the stall. The Connect cluster and the stream processor each add a serialization hop and one more runtime to operate. By the time rows reach Doris, the newest data has already waited on several handoffs.

The Iggy path collapses the middle into a single connector:

Producers -> Apache Iggy -> Rust sink connector -> VeloDB Cloud / Apache Doris
                       ~1 ms write        native Rust, no JVM       Stream Load

Apache Iggy handles ingest on a Rust, io_uring, thread-per-core runtime. Each core runs an independent shard that owns its own partitions, so a core never blocks waiting on another, and there are no JVM garbage-collection pauses. The Rust sink connector reads Iggy topics, batches rows, and posts them to Doris through Stream Load. No Kafka Connect cluster, no stream processor, and no staging store sit in between.

Updates and deletes

Streaming data is full of late arrivals, corrections, and upserts by primary key. Apache Doris resolves them at write time. For a table with a primary key, the connector loads into a Doris Unique Key table with Merge-on-Write, which keeps the latest version of each row as it lands. Reads skip the merge, so query latency stays low while ingest stays high. This is the ingestion pattern the Unique Key model was designed for: the merge cost is paid once at write time, and reads return current state directly.

For append-only event streams, the connector uses a Doris Duplicate Key table, where every event appends a row for fast, ordered scans. Either way, indexes and predicate pushdown apply to the freshest rows, with no separate batch step.

Performance

For operational analytics, fraud detection, AI pipelines, and customer-facing applications, the slowest requests determine user experience. Reducing tail latency means fresher data, faster decisions, and more predictable systems under production load.

The two teams benchmarked Apache Iggy and Apache Kafka as the front-of-pipeline broker for VeloDB Cloud, on matched hardware in a single AWS region, running one broker at a time through the same VeloDB Stream Load path. The full setup, configuration, and run protocol are public in the velodb-demos repository.

Layer 1 measures each broker in isolation, with no database attached:

Apache Iggy delivered roughly 3× the throughput at about 16× lower p99 latency, and held a single-digit-millisecond tail where Kafka climbed toward 80 ms at p99.9, which is about 30× faster. The thread-per-core and io_uring design is what keeps the tail flat under load.

Layer 2 measures the full pipeline, from event to queryable row: event to broker to connector to Stream Load to VeloDB. Here VeloDB is never the bottleneck. Its demo front end commits a Stream Load in about 170 ms with headroom to spare, so end-to-end latency tracks the connector's flush cadence while the database keeps pace. Over a sustained clickstream workload, the Iggy pipeline held a tight tail at p99.9 of 1.3 s, while the Kafka Connect pipeline showed periodic multi-second stalls at p99.9 of 10 s, a tail roughly 7× heavier. Scaled out, VeloDB sustains tens of millions of rows per second, so the broker layer, where Iggy is decisively faster, is the part most worth optimizing.

Try Apache Iggy with VeloDB today

As organizations modernize their data infrastructure, they’re increasingly looking for simpler architectures that deliver higher performance with fewer operational components. The Apache Iggy and Apache Doris integration demonstrates what a fully Rust-native streaming and analytics stack can achieve today—and points toward the next generation of real-time infrastructure.

The Rust sink connector is available now through the Apache Iggy connectors runtime. Point your producers at Iggy, run the sink, and set VeloDB Cloud or your Apache Doris cluster as the target. Your first events can be landing in Doris in minutes.

The complete benchmark, infrastructure, and reproducible run protocol live in the velodb-demos repository: github.com/velodb/velodb-demos/tree/main/iggy

New to VeloDB? Start a free trial of VeloDB Cloud and put sub-second analytics on top of the events you just streamed in.

Try VeloDB Cloud for free: SaaS and BYOC warehouse 30-day free trial with $300 in free credits.

LaserData Cloud is the managed service for Apache Iggy, giving developers a production-ready streaming platform in minutes. Create a cluster, connect your applications, and start streaming events to Apache Doris, VeloDB Cloud, or any supported connector.

Start with the Free Tier, which includes:

• Managed Apache Iggy clusters

• Up to 1 GB/s streaming throughput

• 100 GB of storage

• Built-in observability and management

• Native HTTP, WebSocket, TCP, and QUIC support

Sign up for free at Laserdata.