VARIANT in Apache Doris: Query Billions of JSON Rows and 10K+ Subcolumns in Seconds

Join us for a webinar on Nov. 20 to explore the VARIANT data type in Apache Doris. VARIANT is the key to Apache Doris efficiently storing and querying ultra-wide JSON data, supporting evolving field types and flexible indexing, while maintaining high performance.

JSON has become one of the most widely used data formats: from logs and observability pipelines to e-commerce events and IoT streams. Its flexible, self-describing structure makes it perfect for modern analytics but also challenging to handle efficiently at scale. Many data systems still struggle to query large JSON datasets without sacrificing performance or schema flexibility.

In this session, you'll also see a demo of querying a billion rows of JSON data in seconds with Apache Doris, deployed on an AWS environment.

Key Topics We'll Cover:

1. The Evolution of Semi-Structured Data Analysis: From TEXT and JSON to VARIANT.
2. Challenges of Handling JSON across Data Systems: A look at how Elasticsearch, Snowflake, ClickHouse, and Iceberg manage large-scale JSON, and how Doris tackles the challenge.
3. VARIANT in Doris: New features like sparse columns, subcolumns vertical compaction, and schema templates enable Doris to provide flexible schema, high compression ratio, and fast analytical performance for large-scale JSON data.
4. Demo: Querying 1 billion rows of JSON data in seconds.

Speakers

Owen Xiao Apache Doris PMC and Product VP @ VeloDB

Max Li Senior Engineer @ VeloDB