Briefing
In business scenarios like live streaming and e-commerce, a vast amount of real-time data is generated from various sources. In these scenarios, real-time analysis typically demands the connection of information pieces from multiple streams. Performing multi-stream join operations with a large scale of data in many stream processing systems can lead to the consumption of a substantial amount of computing resources and causing stability issues.
In Apache Doris, Boyang and his teammates could transform the problem of multi-stream join into efficient partial update the engine natively supports. Nevertheless, challenges still persist in areas like upsert flexibility and data lifecycle management. In this slides, Boyang will demonstrate some key features that they developed to enable Apache Doris to address those challenges.
