We at Apache Doris love a good benchmark challenge. So when we first heard about a coffee shop-themed benchmark making waves in the community, we had to try it out.
For background, the Coffee Benchmark was initially designed by Josue "Josh" Bogran, using a synthetic coffee shop dataset to test JOIN-heavy analytical queries between Databricks and Snowflake. The benchmark deliberately includes messy data, imperfect SQL queries, and table JOINS, making it more reflective of the imperfections in real-world production workloads.
We recently ran our first test of the Coffee Benchmark, comparing Apache Doris, ClickHouse, and Snowflake on their respective cloud services. The results were encouraging: on the 721 million rows test, Doris was more than 3x faster than ClickHouse and 6x faster than Snowflake, while running at roughly 1/3 the cost of ClickHouse and 1/5 the cost of Snowflake. More details in our blog
But some in the community raised a fair concern: the underlying hardware setups weren't fully aligned. Because of differences in the cloud server configurations across Doris, ClickHouse, and Snowflake, we couldn't align the compute setup in the first test. And we wanted to know, too, what the results would look like if the resources were fully aligned.
So we went back and re-ran the tests: this time on identical AWS m6i.8xlarge instances, with both Doris (v3.0.7) and ClickHouse (v25.8) deployed in self-managed, storage-compute integrated setups. This ensured that hardware differences no longer influenced the outcome.
Here are the results:
- CoffeeBench: Doris is about 4x faster than ClickHouse
- TPC-H: Doris up to 30x faster
- TPC-DS: Doris up to 40x faster
With the same resource set up, Doris' performance advantage became even more striking. Below, we explain the test environment, methodology, and detailed findings.
1. Test Environment Configuration
- Hardware: 2 × AWS m6i.8xlarge instances, each with 32 vCPUs and 128 GiB of memory.
- Software Versions:
- Apache Doris: Version 3.0.7, deployed on-premises with integrated storage and computing.
- ClickHouse: Version 25.8 (the latest stable release, including recently introduced JOIN operator optimizations), deployed on-premises with integrated storage and computing.
- ClickHouse Cloud: The test data from the previous round is still used in this test as a comparison. Configuration: 2 compute nodes, each with 30 CPUs and 120 GiB of memory. The memory of a single node is slightly lower than that of an m6i.8xlarge instance, likely due to the resource requirements of some other services on the cloud.
2. Test Results
2-1. CoffeeBench 500M
- Apache Doris table creation statements and test statements
- ClickHouse Cloud table creation statements and test statements
- For ClickHouse Local (v25.8), the table creation statements include additional statements for creating distributed tables. The test statements are the same as the ones used in ClickHouse Cloud.
Looking at total query runtime, Apache Doris showed a clear performance advantage: Apache Doris was 3x faster than ClickHouse v25.8 and 2.6x faster than ClickHouse Cloud. Across all 17 queries in this test, Doris consistently delivered the fastest results, demonstrating both stability and strong overall performance.
Detailed Test Results:
2-2. CoffeeBench 1B
The table creation statements and test statements were the same as in CoffeeBench 500M.
Total runtime again showed Doris's strength: Apache Doris was 4x faster than ClickHouse v25.8 and 2.5x faster than ClickHouse Cloud. Out of 17 individual query tests, Doris achieved the fastest execution in 16 of them (more than 94% of the workload), showing a consistent performance lead.
Detailed Test Results:
2-3. TPC-H 100GB
The Coffee Benchmark is helpful for testing joins, but its 2–3 table JOINS are still too simple to reflect real-world analytics. To capture more realistic workloads with multiple table joins, nested subqueries, and complex aggregations, we also ran TPC-H and TPC-DS on 100 GB datasets, which are two widely used OLAP benchmarks, providing a broader measure of database performance for demanding real-world analytical scenarios.
- Apache Doris table creation statements and test statements
- ClickHouse Cloud table creation statements and test statements
- For ClickHouse Local (v25.8), the table creation statements include additional statements for creating distributed tables. The test statements are the same as the ones used in ClickHouse Cloud.
Results: Under the same resource settings, Apache Doris outperformed ClickHouse by a wide margin: Doris is 30x faster than ClickHouse v25.8 and 12x faster than ClickHouse Cloud. Doris led in all 22 queries, while ClickHouse Local failed to complete 4 of them. On the latency front, most Doris queries finished within about 200ms, delivering a strong guarantee for real-time, service-facing workloads.
2-4. TPC-DS 100g
- Apache Doris table creation statements and test statements
- ClickHouse Local (v25.8) table creation statements (on-premises)
- ClickHouse Cloud table creation statements
- ClickHouse query statements
Looking at total query time, Apache Doris demonstrated a decisive performance advantage: Apache Doris was 40x faster than ClickHouse v25.8 on-premises and 30x faster than ClickHouse Cloud. Across 99 individual queries, Doris outperformed ClickHouse on 97 of them: an overwhelming 98% win rate, showing superior performance.
Detailed Test Results:
Conclusion
In every benchmark tested: CoffeeBench, TPC-H, and TPC-DS, Apache Doris consistently pulled ahead, establishing clear dominance over both ClickHouse v25.8 on-premises and ClickHouse Cloud.
- CoffeeBench (17 standard queries): Doris was fastest on 16 queries, running 4x faster than ClickHouse v25.8 and 2.5x faster than ClickHouse Cloud.
- TPC-H (22 complex queries): Doris outperformed across all queries, achieving 30x the speed of ClickHouse v25.8 and 12x the speed of ClickHouse Cloud, with the latter failing four queries.
- TPC-DS (99 high-complexity queries): Doris delivered 40x faster performance than ClickHouse v25.8 and 30x faster than ClickHouse Cloud, winning 97 out of 99 queries.
More importantly, Doris' strength lies not only in handling high-frequency, moderately complex queries efficiently, but also in its architectural advantages in query planning and resource scheduling. Even under workloads with heavy joins and complex business logic, Apache Doris consistently delivered most queries within the 200 ms range, ensuring smooth online service experiences and showing its advantages in large-scale data analytics.
Want to learn more about Apache Doris and its real-time analytics performance in benchmarks? Join the Apache Doris community on Slack and connect with Doris experts and users. If you're looking for a fully-managed, cloud-native version of Apache Doris, contact the VeloDB team.
