Canva was taking a full snapshot of every service database every 24 hours to feed a 5-petabyte Snowflake warehouse. When a snapshot started taking longer than 24 hours to extract, the whole schedule fell apart.

Canva’s data analytics platform stores over 5 petabytes — 5,000 terabytes — in Snowflake, fed from a growing number of internal first-party services. To get service data into that warehouse, the platform used snapshot replication: take a full snapshot of each service database every 24 hours, drop it into an S3 bucket, then run a scheduled job to ingest those files into the warehouse.

It was simple and it worked, right up until the databases got big. As services stored more and more data, the time it took to extract a full snapshot kept climbing.

Two different problems were tangled together. The first was throughput: the extraction process needed to stop scaling with the total size of the database and start scaling with how much the data actually changed. A daily snapshot re-copies everything even when almost nothing moved.

The second was ownership. As Canva added services, it wasn’t enough to make extraction faster — the platform needed a way for the teams that produce data to own its pipeline end to end, rather than routing every change through a central data team.

The technical answer was change data capture. Instead of snapshotting the entire database, CDC tracks Data Manipulation Language changes — inserts, updates, and deletes — and streams them as JSON records to a target system, which rebuilds the current state of the source. Because the network only ever carries what changed, the volume is far smaller than a daily full copy, and it arrives as a continuous trickle instead of one enormous daily spike.

Canva wired this up per source type: MySQL databases use the continuous-replication feature of AWS DMS, and DynamoDB sources use Kinesis Data Streams for DynamoDB. The change records flow through Kinesis Data Firehose into a partitioned S3 bucket, and Snowflake’s Snowpipe continuously ingests them into the warehouse.

Solving throughput exposed the second problem: who owns each service’s data and pipeline. Canva’s answer is a service-aligned architecture with data-mesh-like characteristics — instead of splitting ownership by technology, resources are grouped by the domain of the service that produces them. Using Infrastructure as Code and common modules built by the data platform team, service owners stand up and own their own end-to-end data infrastructure.

In the warehouse, each service’s data is structured in three layers: a source layer of raw replicated data, a model layer with the service owner’s business logic applied, and an expose layer of data published for downstream consumers. dbt orchestrates the load and transform pipelines that move data between them.

The move gave Canva what it needed: CDC lets updates land in the warehouse within minutes instead of on a 24-hour cycle, and the higher velocity gives the team far more flexibility in scheduling load and transform jobs, since they’re no longer gated by the snapshot extraction. The service-aligned model pushes pipeline ownership out to the domains that generate the data.

Canva is candid that this is a journey, not a finished result. They migrated CDC to the services that most urgently needed off snapshots, but rolling the pattern out to the rest is still ongoing, and they note open questions around richer orchestration and observability of the mesh as more services and dependencies join it.

A periodic full-copy pipeline isn’t slow until the day one copy takes longer than the interval between copies — after that it’s simply broken, and the fix is to move only what changed.