
One AI agent can run more queries in an hour than an analyst team can in a week, and agents often demand fresh events and full history in a single answer. Supporting agent workloads means running two pipelines at once: batch and real-time. Most teams join those two with glue code, and face maintenance overhead.
Together with dltHub, Tower.dev, we will demo how to build batch + real-time pipelines with a coding agent (Claude Code here, though any agent works) and run them in production. You'll see firsthand what serving takes when agents are the primary consumers.
Background: dlt is an open-source Python library for data ingestion, with 5,000+ sources supported. Tower is the serverless runtime and orchestration layer those pipelines run on, with scheduling, monitoring, and a managed Iceberg lakehouse. VeloDB is a real-time analytics and search database that serves the results, with sub-second latency at thousands of concurrent queries and hybrid search for agent context retrieval.
Demos:
-
Using Claude Code and dlt to write an ETL job from scratch, loading the resulting tables into S3, and querying them with VeloDB. dlt handles ingestion and structure, VeloDB runs the joins and aggregations that get harder as the dataset grows.
-
A dlt pipeline deployed on Tower, landing batch data in Iceberg and streaming events into VeloDB, queried by an agent seconds after ingestion.
Speakers:
Kevin Shen, Principal Product Manager, VeloDB
Elvis Kahoro, Developer Experience and Ecosystem Lead, dltHub
Vim Wickramasinghe, Founding Software Engineer, Tower.dev