PostgreSQL CDC to Apache Doris: Real-Time Sync with One SQL statement

Apache Doris syncs PostgreSQL data in real time through built-in change data capture (CDC). One CREATE JOB statement sets up the whole pipeline: Doris creates the target tables from the PostgreSQL schema, loads the full history, and streams incremental changes from the PostgreSQL write-ahead log (WAL). No Debezium, Kafka, Flink, or Spark sits in the path; the sync is a single hop from PostgreSQL to Apache Doris.

TL;DR

Built-in PostgreSQL CDC runs change capture, task scheduling, data writing, and checkpoint-based resume inside the Apache Doris Job scheduling framework.
One SQL statement creates the sync job. Doris creates the target tables, loads full historical data, then follows the WAL continuously.
Monitoring is one query: jobs(type='insert') reports CurrentOffset (the sync position) and LoadStatistic (rows and bytes written).
Built-in CDC fits pure table sync. A dedicated pipeline (Debezium or Flink CDC, Kafka, Flink or Spark) still fits when changes need routing, enrichment, or transformation in transit.

Why sync PostgreSQL into an analytical database?

Analytical queries strain PostgreSQL. PostgreSQL was designed for high-concurrency online transaction processing (OLTP) and runs the transactional core of a wide range of business systems, so cross-table aggregations and large scans can consume enough system resources to affect the stability of the online business that shares the database.

The common practice is to sync PostgreSQL data in real time into an analytical database such as Apache Doris, then point real-time analytics, live dashboards, data warehousing, and AI applications at that copy. Doris handles the analytical queries, and the transactional and analytical workloads stay separate.

Building and maintaining that sync pipeline is usually harder than running the analytics system itself. Moving PostgreSQL data into Doris continuously, stably, and accurately is the critical link in the whole architecture.

Challenges of the traditional sync pipeline

A traditional PostgreSQL-to-Doris pipeline chains three independent systems between the two databases, and each one adds its own operational work. Figure 1 shows the pipeline next to the built-in CDC approach in Apache Doris; the built-in approach is what the rest of this post covers.

A traditional sync pipeline has three parts:

Change capture. A CDC component such as Debezium or Flink CDC continuously captures data changes from the PostgreSQL write-ahead log (WAL).
Message delivery. Depending on business needs, changes are written to Kafka or sent directly to downstream systems.
Data processing. A compute framework such as Flink or Spark transforms and cleans the data, then writes the results to Apache Doris.

That architecture is flexible and handles complex processing logic. It also chains together several independent systems, each with its own maintenance work. When the requirement is simply moving data from PostgreSQL to Doris in real time, two problems follow:

Many components, complex deployment. Beyond the databases themselves, the pipeline needs CDC, Kafka, and Flink or Spark, each with its own deployment, monitoring, upgrade, and scaling work. For a team that only needs data sync, that is a lot of infrastructure to own.
A long pipeline, difficult troubleshooting. When data arrives late, goes missing, or looks wrong, the fault has to be traced through the CDC layer, the message queue, and the compute engine one at a time. Locating the problem is slow.

Both problems come from the components sitting between PostgreSQL and Doris.

How built-in PostgreSQL CDC works in Apache Doris

Apache Doris removes the middle layer of CDC tool, message queue, and compute engine. Change capture, task scheduling, data writing, and checkpoint-based resume all run inside the Doris Job scheduling framework, so there is nothing separate to deploy and the sync path simplifies to a single hop: PostgreSQL → Apache Doris.

The systems to maintain shrink from a set of independent components to one Doris cluster. Creating a sync job takes one SQL statement, and Doris handles everything downstream of it: target table creation, the full historical load, and continuous incremental sync.

Three components keep sync tasks moving continuously: the frontend (FE) schedules the work, the backend (BE) executes it, and a CDC Client pulls change data from PostgreSQL.

The execution flow:

A user creates a PostgreSQL streaming job in the FE through SQL, specifying the source database, tables, and related parameters.
The FE creates the Doris target tables from the PostgreSQL schema, if they do not already exist.
The Job Scheduler generates a sync task and dispatches it to a BE node.
The BE forwards the request to the CDC Client on the same machine, starting the client first if it is not already running.
The CDC Client reads the PostgreSQL WAL continuously, collecting both the full historical data and incremental changes.
The data is serialized and written into Doris through Stream Load.
When the current batch finishes writing, the BE reports the sync offset to the FE.
The FE persists the offset and schedules the next round of sync tasks, which keeps the sync running continuously.

Data sync, task scheduling, and state management all stay inside Doris, with no dependency on Kafka, Flink, or Spark. The path is shorter, the scheduling reuses a framework Doris already runs, and there is one system to check when the sync falls behind.

Quick start: create a sync job

The example below creates a sync job that replicates the tables student1 and student2 from a PostgreSQL schema into the Doris database target_test_db:

CREATE JOB test_postgres_job
ON STREAMING
FROM POSTGRES (
  "jdbc_url" = "jdbc:postgresql://127.0.0.1:5432/postgres",
  "driver_url" = "postgresql-42.5.0.jar",
  "driver_class" = "org.postgresql.Driver",
  "user" = "postgres",
  "password" = "postgres",
  "database" = "postgres",
  "schema" = "cdc_test",
  "include_tables" = "student1,student2",
  "offset" = "initial"
)
TO DATABASE target_test_db

Once created, the job completes the rest on its own:

creates the target tables in Doris, matching the PostgreSQL schema;
loads the full historical data;
consumes the PostgreSQL WAL continuously, syncing incremental changes in real time.

How to check sync job status

One query shows the job’s health and progress. The output here is trimmed to the fields worth watching:

mysql> select * from jobs(type='insert') where name='test_postgres_job'\G
*************************** 1. row ***************************
Name: test_postgres_job
ExecuteType: STREAMING
Status: RUNNING
SucceedTaskCount: 3
FailedTaskCount: 0
CurrentOffset: {"lsn":"53547640","txId":"1523","splitId":"binlog-split"}
LoadStatistic: {"scannedRows":7,"loadBytes":341,"fileNumber":0,"fileSize":0,...
JobRuntimeMsg: No data available for consumption at the moment, will retry...

Two fields matter most. CurrentOffset records the current sync position, and LoadStatistic reports the rows and bytes written. Read together, they show whether the job is healthy and how far the sync has progressed.

For the full parameter reference and advanced usage, see PostgreSQL CDC with auto table creation in the Apache Doris documentation.

When should you use built-in CDC?

Built-in CDC fits the case where sync is the whole requirement: keep a set of PostgreSQL tables current in Doris, with no transformation along the way. One SQL statement sets it up, Doris creates the target tables, loads the history, and follows the WAL from there, and a single system covers the whole path.

A dedicated CDC stack still earns its place when the pipeline has to do real work in transit: routing changes to several destinations, joining or enriching streams, or applying transformation logic before the data lands. Purpose-built CDC tools are improving quickly, and they remain the right answer for that shape of problem.

Requirement	Better fit
Keep PostgreSQL tables current in Doris, no transformation	Built-in Apache Doris CDC
Route changes to several destinations, join or enrich streams, or transform data before it lands	Dedicated pipeline (Debezium or Flink CDC, Kafka, Flink or Spark)

Frequently asked questions

Does built-in Doris CDC need Kafka, Flink, or Debezium?

No. Change capture, task scheduling, data writing, and checkpoint-based resume all run inside the Apache Doris cluster, with no separate CDC tool, message queue, or compute engine to deploy.

Does Doris create the target tables automatically?

Yes. The FE creates the Doris target tables from the PostgreSQL schema if they do not already exist, loads the full historical data, then syncs incremental changes continuously.

How does a sync job recover after an interruption?

The FE persists the sync offset each time a batch finishes writing, and checkpoint-based resume restarts the job from that persisted offset.

Availability

Built-in PostgreSQL CDC ships in Apache Doris 4.1 and is currently marked experimental in the official documentation. On the source side, PostgreSQL 14 or later is supported, with logical replication enabled (wal_level = logical, or rds.logical_replication = 1 on Amazon RDS) and a publication named dbz_publication created FOR ALL TABLES. Only tables with primary keys can be synced.

The same capability is also available via VeloDB, the commercial distribution of Apache Doris, where it will further reduce the cost of running PostgreSQL real-time sync in production. To run the same sync as a managed service, see VeloDB Cloud.