Emulated queues on custom objects⟶Platform Events — a Kafka-inspired time-ordered event log
Without native eventing, Salesforce customers were emulating message queues with custom objects — fragile, slow, and billed as extra storage. No existing broker pattern could scale to hundreds of thousands of tenants, so Salesforce borrowed Kafka's log instead of a queue.
A private queue per subscriber doesn't scale to hundreds of thousands of tenants — an immutable log that subscribers read at their own pace does.
No native events, so customers built their own out of custom objects
Salesforce’s author on this post joined the company five years earlier with a charter to improve its integration capabilities. Back then, moving data in or out of Salesforce meant the REST and SOAP APIs, and business-process integration meant making web service calls via Apex — nothing asynchronous, nothing event-driven.
Customers wanted it anyway. Talking to customers and ISVs, Salesforce found they were already building asynchronous applications on the platform — just without any native support for it, “they had to emulate queueing with custom objects.”
REQUIREMENTS
Pub/sub, with durability, at multitenant scale
Salesforce settled on a publish/subscribe model as the right shape — more flexible than point-to-point messaging, since a pub/sub topic with one subscriber can behave like a queue, but a pure queue can’t be exposed to multiple independent consumers the way a topic can. On top of that, they wanted subscription durability: the ability to persist in-flight messages for a period of time, so the system worked for more than just transient, ephemeral use cases.
OPTIONS
The standard broker pattern assumes one queue per subscriber
Traditional durable pub/sub, over protocols like AMQP, achieves durability through individual message acknowledgement — which in practice means every durable subscriber needs its own private queue on the broker. That’s a workable design when you have dozens or hundreds of subscribers. Salesforce’s platform deals with tenants “that number in the hundreds of thousands, each of which may have multiple subscribers,” and standard routing-configuration-file approaches don’t work either, because routing can be different per tenant. As the team put it, “there’s no (economical) way to spin up a new set of brokers for each individual tenant.” They concluded it wasn’t feasible to directly reuse any existing message broker product or framework.
THE CALL
Borrow Kafka's log, not a queue at all
Apache Kafka was gaining traction inside Salesforce around the same time, and its model was a structural fit rather than an incremental improvement: Kafka doesn’t use queues, it arranges events as an immutable, time-ordered log. Publishers append to the head of the log; each event gets an offset that fixes its order; subscribers read at their own pace and can resume from their last-seen offset after a disconnect, with old events periodically purged to bound storage.
That shape solves the multitenant problem directly: each message is stored on disk once, no matter how many subscribers read it, and tracking a subscriber’s read position is the subscriber’s job, not the broker’s — so the broker never accumulates state per subscriber the way a private-queue design does. But the log only solves “temporal decoupling” — publishers and subscribers not needing to be online at the same moment. It says nothing about whether they agree on the shape of the message. For that, Salesforce reused its own metadata repository: events are defined as custom entities, published as schemas subscribers can discover independently, and translated to Apache Avro for efficient cross-language serialization. That adds logical decoupling and, through the same discovery mechanism, lifecycle decoupling — publishers and subscribers can evolve without depending on each other’s implementations.
CONSEQUENCES
An enterprise event layer that lives inside the metadata system it borrowed
The result, Platform Events, combines a time-ordered log with Salesforce’s metadata layer: events are defined like custom objects, discoverable through Setup, the Developer Console, Describe, Tooling, or the Metadata API. Internally, Apex triggers, Process, and Visual Flow can publish or subscribe; externally, topics are exposed through the Salesforce Streaming API, and any SObject-capable API can publish, since “Platform Events are first-class Salesforce Objects after all.”
The trade-off Salesforce accepted, stated plainly rather than hidden: a subscriber that falls behind for longer than the retention period can miss events outright, because the log purges the oldest entries to conserve resources. Durability here is bounded, not absolute — the price of a design that scales to hundreds of thousands of tenants without hundreds of thousands of private queues.
When a messaging pattern that works fine at ordinary scale breaks down at yours, the fix usually isn’t a bigger version of the same pattern — it’s borrowing a different one wholesale, in this case swapping a per-subscriber queue for a log the subscriber has to actively keep up with.
Two different relationships with Kafka — Twitter eventually adopted it outright, Salesforce borrowed its log model to build something Kafka-shaped without running Kafka itself.
A plain-language, AI-drafted and human-edited retelling of the article published on engineering.salesforce.com, reorganized and explained in our own structure and words, with original analysis in the editor's note above. The facts, numbers, and decisions belong to the original author and are not altered. For the full depth, read the source.