Exactly-Once Delivery is Impossible

Background

We discussed this in previous articles that when engineers first encounter distributed systems, they subconsciously carry an assumption from single-machine programming: if something happens, the system knows it happened. If it didn’t happen, the system knows that too. This assumption is rarely stated out loud, but it shows up everywhere—in how we design APIs, how we reason about failures, and how confident we are when we say things like “the request failed.”

We dismantled this assumption as well already. We saw that timeouts do not tell us what happened, only that we waited long enough to become uncomfortable. We saw that retries are not sloppy engineering, but a rational response to uncertainty. Once a system operates across processes, machines, or networks, it can no longer observe reality directly. It must act based on incomplete information.

Once this is accepted, a deeper question naturally follows: if systems cannot know with certainty whether an operation happened, how can they ever guarantee that it happened exactly once?

This is where the idea of exactly-once delivery enters the conversation. It is one of the most attractive promises in distributed systems, and can easily be one of the most misunderstood.

Exactly-once delivery sounds like the ideal world. A message is sent once, delivered once, processed once, and never duplicated. Nothing is lost, nothing is repeated, and downstream systems can stay simple. If such a guarantee were possible in the general case, an enormous amount of complexity would disappear. There would be no need for deduplication or idempotency. We could trust the infrastructure to “just handle it.”

It is therefore not surprising that many of us believe this should be achievable. After all, databases often feel exactly-once. Function calls feel exactly-once. Even some distributed systems documentation uses the phrase casually. But this intuition quietly assumes something that no distributed system can have: perfect knowledge of what is occurring.

To talk about this clearly, it helps to introduce the standard delivery semantics used in distributed systems: at-most-once, at-least-once, and exactly-once. These terms represent fundamentally different tradeoffs.

At-most-once delivery

As the name suggests, it means a message will be delivered at most once or won’t be delivered at all. If something goes wrong, the system does not intentionally retry. The message may simply be lost. This model aligns closely with the simplest possible behavior: send a request, wait for a response, and if nothing arrives, move on. No retries, no tracking, no recovery.

Note: Technically, some retries by default can happen from clients or load balancers, but the system won’t retry intentionally.

The appeal of at-most-once delivery is that duplication is impossible. Because the system never retries, downstream logic never has to worry about processing the same message twice. There is no risk of double-charging a user or duplicating a record due to retries. The system is simple, fast, and predictable.

The cost of this simplicity is correctness. If a request reaches a server, is partially processed, and the response never makes it back to the caller, the system has no way to recover. At-most-once delivery assumes that occasional loss is acceptable. This is why it is better to be used for telemetry, metrics, logging, and other best-effort data. If a log line or metric point is dropped, no invariant is violated. The system still functions.

At-least-once delivery

At-least-once delivery sits on the other side of the spectrum. Under this model, the system guarantees that a message will eventually be delivered, even if it has to be delivered multiple times. Loss is unacceptable, so retries are mandatory.

This model emerges naturally once uncertainty is acknowledged. As discussed in previous articles, if a system cannot know whether a message was processed successfully, and correctness matters, the only rational response is to retry. Over time, this guarantees delivery, but it introduces duplication. The same message may be processed twice, concurrently, or long after the original attempt.

Most real-world distributed systems choose at-least-once delivery precisely because it aligns with reality. Load balancers retry. Clients retry. Message queues retry. Even infrastructure you don’t control retries on your behalf.

At-least-once delivery preserves correctness in the face of failure, but only if the system is designed to tolerate repetition. Without additional safeguards, duplication leads to corrupted state: double charges, duplicate orders, repeated notifications, and inconsistent counters. This is why at-least-once delivery must always be paired with idempotent processing. Delivery is allowed to repeat; effects are not.

Exactly-once delivery

Exactly-once delivery appears, at first glance, to offer the best of both worlds: no loss and no duplication. The reason it is so tempting is that it seems like a small step beyond at-least-once. Add acknowledgements. Track offsets. Retry carefully. Surely, with enough bookkeeping, the problem can be solved.

The flaw in this reasoning is not in the bookkeeping, but in the assumption that acknowledgements represent truth.

An acknowledgement is also a message.

It can be delayed, duplicated, reordered, or lost in exactly the same ways as the original message. If a sender does not receive an acknowledgement, it cannot tell whether the message failed or succeeded. This is just another variant of the two generals’ problem we discussed previously. What matters is the consequence: retries are unavoidable, and retries imply possible duplication.

This leads to a fundamental conclusion that we need to eventually internalize: exactly-once delivery in the general case is impossible. Not difficult. Impossible. It would require perfect knowledge of events in an imperfect world.

This is why serious systems stop trying to guarantee delivery and instead focus on something more achievable: guaranteeing effects.

Exactly-once delivery may be impossible; its effects are not

Delivery is about movement. It is about messages traveling through space and time. Effects are about state. They are about whether money was charged, whether a record was created, whether a balance was updated. Delivery can be duplicated. Effects must not be allowed to accumulate incorrectly.

Once this distinction is made, the design goal shifts. The system no longer tries to prevent messages from arriving more than once. Instead, it ensures that processing the same message multiple times produces the same final state. Repetition is no longer an error; it is a condition to be handled.

This is what we discussed about in idempotency. What idempotency means is exactly-once effects. A message might be delivered ten times. The system ensures that the business outcome occurs once.

This reframing explains why many products have, at various points, claimed to offer exactly-once guarantees without actually contradicting reality.

How products promise exactly-once delivery

Kafka’s “exactly-once semantics” is a good example. Kafka does not guarantee exactly-once delivery across arbitrary consumers, failures, and side effects. What it guarantees is exactly-once processing within a carefully defined boundary.

In Kafka’s case, that boundary is a single Kafka cluster with transactional producers and consumers, where state changes are coordinated with offset commits. A consumer processes records, updates its state, and commits offsets as part of a single transaction. If the consumer crashes before the transaction commits, the transaction is aborted and the records are replayed. If the transaction commits, both the state update and the offset commit become visible together.

Within this boundary, effects appear exactly once.

The hidden condition is crucial: all relevant state must participate in the same transactional system. As soon as side effects escape that boundary—sending an email, calling an external API, charging a credit card—the guarantee no longer applies. So, Kafka did not solve exactly-once delivery in the open world. It solved exactly-once effects inside a controlled domain with a trusted coordinator.

Databases make similar promises. When a database executes a unique transaction, it guarantees that the transaction’s effects are applied once, even if the client retries after a timeout. This works not because the request arrived once, but because the database enforces atomicity and uniqueness. The database is the authority that decides whether a state change is allowed to occur.

Conclusion

In every real system that claims exactly-once behavior, this same pattern appears. The guarantee is never about delivery itself. It is about effects within a boundary, enforced by a system that has the authority to reject duplicates and serialize conflicts. Once you leave that boundary, the guarantee dissolves.

This is why one needs to be skeptical of broad exactly-once claims. Without precise definitions, the promise is usually illusory.

This conclusion is not pessimistic. It is clarifying. Exactly-once delivery is the wrong goal. Exactly-once effects are achievable, practical, and sufficient. Systems that embrace this reality are easier to recover, easier to reason about, and more honest about their limitations.

Now, if delivery cannot be trusted, then correctness must come from somewhere else. Understanding how systems decide on correctness, requires talking about consistency and authority. That is where we go next.