Store-and-Forward vs Fire-and-Forget Messaging

Store-and-Forward vs Fire-and-Forget Messaging in one line: fire-and-forget sends a message and moves on without waiting or keeping a copy, so a message sent while the receiver is down is gone; store-and-forward writes the message durably at an intermediary before the sender gets its ack, then delivers it whenever the receiver is back. The first is cheap and lossy; the second pays in latency, storage and duplicates. Most systems use both.

Side by side:

Style	When the receiver is down	What it costs
Fire-and-forget	the message is lost	almost nothing; no ack, no storage
Store-and-forward (broker)	kept, delivered after recovery	a durable write per message, possible duplicates
Outbox (your own database)	kept in a table, relayed later	a table, a relay, possible duplicates

The problem it solves

Either side of a message can be down when it is sent. The design question is what happens to a message sent into that gap:

• A metric sample, a debug log line, a heartbeat. The next one arrives in a second. Losing a few is fine, and waiting for an ack would slow the sender for nothing.
• An order, a payment, a password-reset email. Losing one is a bug report or a refund. The sender must know the message is safe somewhere before it tells its own caller "done".

One style for everything is the mistake: fire-and-forget for orders loses money, and store-and-forward for every metric sample protects numbers nobody would miss.

The intuition

Fire-and-forget is a postcard dropped in a box with no tracking. A UDP datagram is the textbook case, and so is a metrics agent sending samples over UDP or a log call into a buffer. The sender learns nothing, not even whether the receiver was running.

Store-and-forward is a registered letter left at the post office. The intermediary, a message broker, a mail relay or a table in your database, writes the message durably first, then acks the sender. From that moment the sender is free; the intermediary owns delivery. It forwards when the receiver is up, and deletes its copy only when the receiver acks.

That last step is where duplicates come from. If the receiver processes a message and the ack is lost, or the intermediary crashes before recording it, the copy is still stored and gets sent again. Store-and-forward gives at-least-once delivery, so the receiver has to be idempotent: it keeps the ids of messages it has applied and ignores a second copy. The sibling article on message queues covers acks, redelivery and dead letters in depth.

The outbox pattern is store-and-forward inside your own database. Updating a row and publishing an event is a dual write: commit the row, crash, and the event never goes out. Instead, write the event into an outbox table in the same transaction as the row. A relay reads unsent outbox rows, publishes them and marks them sent. The row and the message now commit together or not at all, and a crash in the relay only causes a resend.

Watch it run

The animation puts a queue between the producer and the worker, which is the store-and-forward shape. The producer writes a message and returns at once; the worker pulls it when it has capacity and acks, and only then is the message removed. A burst arrives and nothing times out: jobs wait, stored and visible, until workers drain them. Then the guarantees: the same message can arrive twice, so processing it twice must be harmless. Fire-and-forget is everything this picture removes: no queue, no ack, no second chance.

The code

Derived from the lesson's queue model: a producer, a stored copy, a consumer that acks. One message per tick for 30 ticks, and the consumer is down for ticks 10 to 19. Fire-and-forget sends straight to the consumer; store-and-forward writes to a broker first, and the broker deletes its copy only on ack:

from collections import deque

class Consumer:
    """The worker side of the lesson's queue. It can be down; it may dedupe by key."""
    def __init__(self, down=range(0), idempotent=False):
        self.down, self.idempotent = down, idempotent
        self.applied, self.seen, self.duplicates = [], set(), 0

    def up(self, tick):
        return tick not in self.down

    def receive(self, msg_id, body):
        if msg_id in self.seen:
            self.duplicates += 1                      # the same message, a second time
            if self.idempotent:
                return                                # absorbed: the key was already applied
        self.seen.add(msg_id)
        self.applied.append(body)

def fire_and_forget(consumer, ticks):
    """Send straight to the consumer. No ack, no copy kept: if it is down, the message is gone."""
    lost = 0
    for t in ticks:
        if consumer.up(t):
            consumer.receive(t, "order-%d" % t)
        else:
            lost += 1                                 # nobody stored it, nobody will retry
    return {"delivered": len(consumer.applied), "lost": lost,
            "duplicates": consumer.duplicates, "writes": 0, "max_wait": 0}

class Broker:
    """Store-and-forward: write durably, ack the producer, forward when the consumer is up."""
    def __init__(self):
        self.stored, self.writes, self.max_wait = deque(), 0, 0

    def accept(self, msg_id, body):
        self.stored.append((msg_id, body))            # durable write BEFORE the producer's ack
        self.writes += 1
        return "ack"

    def forward(self, consumer, tick, crash_before_ack=()):
        while self.stored and consumer.up(tick):
            msg_id, body = self.stored[0]
            consumer.receive(msg_id, body)            # the consumer has done the work...
            self.max_wait = max(self.max_wait, tick - msg_id)   # ticks spent in storage
            if msg_id in crash_before_ack:
                crash_before_ack.discard(msg_id)      # ...and the broker dies before the ack
                return                                # the message is still stored: resent
            self.stored.popleft()                     # ack received: delete the copy
            self.writes += 1

def store_and_forward(consumer, ticks, crash_before_ack=()):
    broker, crashes = Broker(), set(crash_before_ack)
    for t in ticks:
        broker.accept(t, "order-%d" % t)              # the producer is done after this ack
        broker.forward(consumer, t, crashes)
    later = max(ticks[-1] + 1, consumer.down.stop)    # the consumer is back by now
    while broker.stored:
        broker.forward(consumer, later, crashes)      # drain what is still stored
    return {"delivered": len(set(consumer.seen)), "lost": len(broker.stored),
            "duplicates": consumer.duplicates, "writes": broker.writes, "max_wait": broker.max_wait}

ticks = range(30)                                     # one message per tick
down = range(10, 20)                                  # the consumer is down for ten ticks
print(fire_and_forget(Consumer(down), ticks))
# {'delivered': 20, 'lost': 10, 'duplicates': 0, 'writes': 0, 'max_wait': 0}
print(store_and_forward(Consumer(down), ticks))
# {'delivered': 30, 'lost': 0, 'duplicates': 0, 'writes': 60, 'max_wait': 10}
naive, safe = Consumer(down), Consumer(down, idempotent=True)
print(store_and_forward(naive, ticks, crash_before_ack={25}), len(naive.applied))
# {'delivered': 30, 'lost': 0, 'duplicates': 1, 'writes': 60, 'max_wait': 10} 31
print(store_and_forward(safe, ticks, crash_before_ack={25}), len(safe.applied))
# {'delivered': 30, 'lost': 0, 'duplicates': 1, 'writes': 60, 'max_wait': 10} 30

Fire-and-forget lost exactly the ten messages sent into the outage. Store-and-forward delivered all 30, for 60 storage writes (store, then delete) and waits of up to 10 ticks. The crash before the ack delivered message 25 twice: the naive consumer applied 31 orders, the idempotent one 30.

The outbox, on SQLite. The order row and its message commit together; the lamp order crashes mid-transaction and leaves neither behind. The relay then dies after sending the mug and before marking it, so the next run sends it again:

import sqlite3

db = sqlite3.connect(":memory:", isolation_level=None)       # we issue BEGIN/COMMIT
db.executescript("""
CREATE TABLE orders (id INTEGER PRIMARY KEY, item TEXT NOT NULL);
CREATE TABLE outbox (id INTEGER PRIMARY KEY, msg_id TEXT UNIQUE NOT NULL,
                     body TEXT NOT NULL, sent INTEGER NOT NULL DEFAULT 0);
""")

def place_order(item, crash=False):
    """The order row and its message commit together, or neither does."""
    db.execute("BEGIN")
    oid = db.execute("INSERT INTO orders (item) VALUES (?)", (item,)).lastrowid
    if crash:
        db.execute("ROLLBACK")                       # the process died mid-transaction
        return
    db.execute("INSERT INTO outbox (msg_id, body) VALUES (?, ?)", ("order-%d" % oid, item))
    db.execute("COMMIT")

def relay(consumer, crash_before_mark=()):
    """Forward unsent outbox rows, then mark them. A crash in between means a resend."""
    for row_id, msg_id, body in db.execute(
            "SELECT id, msg_id, body FROM outbox WHERE sent = 0 ORDER BY id").fetchall():
        consumer.receive(msg_id, body)
        if msg_id in crash_before_mark:
            crash_before_mark.discard(msg_id)
            return                                   # stopped before the UPDATE
        db.execute("UPDATE outbox SET sent = 1 WHERE id = ?", (row_id,))

shop = Consumer(idempotent=True)
for item, crash in [("book", False), ("lamp", True), ("mug", False), ("pen", False)]:
    place_order(item, crash)
relay(shop, crash_before_mark={"order-2"})           # dies after sending the mug
relay(shop)                                          # the next run resends it
count = lambda sql: db.execute(sql).fetchone()[0]
print(count("SELECT COUNT(*) FROM orders"), count("SELECT COUNT(*) FROM outbox"),
      count("SELECT COUNT(*) FROM outbox WHERE sent = 0"))
# 3 3 0
print(shop.applied, shop.duplicates)                 # ['book', 'mug', 'pen'] 1

The seeded check: 500 runs with random lengths, outage windows and up to three crashes before the ack. Fire-and-forget must lose exactly the messages sent while the consumer was down; store-and-forward must lose none; the naive consumer must see one duplicate per crash; the idempotent one must apply every message exactly once, in order:

import random

rng = random.Random(14)
ok, lost_total = True, 0
for _ in range(500):
    n = rng.randint(1, 60)
    start = rng.randrange(n)
    down = range(start, start + rng.randint(0, 20))
    crashes = set(rng.sample(range(n), rng.randint(0, min(3, n))))
    ff = fire_and_forget(Consumer(down), range(n))
    ok &= ff["lost"] == len([t for t in range(n) if t in down])
    ok &= ff["delivered"] + ff["lost"] == n
    naive, safe = Consumer(down), Consumer(down, idempotent=True)
    sf_naive = store_and_forward(naive, range(n), crashes)
    sf_safe = store_and_forward(safe, range(n), crashes)
    ok &= sf_naive["lost"] == sf_safe["lost"] == 0
    ok &= sf_naive["duplicates"] == len(crashes) == len(naive.applied) - n
    ok &= safe.applied == ["order-%d" % t for t in range(n)]   # each once, in order
    lost_total += ff["lost"]
print(ok, lost_total)                                # True 3407

The complexity

• Fire-and-forget: one send per message, no waiting on the receiver, no storage. Loss grows with outage length.
• Store-and-forward: one durable write before the ack, one delete or mark after delivery, and storage for everything sent during an outage. Producer latency includes the durable write.
• Delivery delay: up to the outage length plus the time to drain the backlog.
• Idempotency: one stored key per applied message, kept at least as long as redelivery is possible.

Where it goes wrong

• An in-memory retry buffer called "reliable". If the sender crashes, the buffer dies with it. Store-and-forward means durable storage that outlives the sender.
• Acking the producer before the write is durable. Then it is fire-and-forget with extra steps.
• Dual writes instead of an outbox. Commit the row, crash before publishing, and the event is gone with no error.
• Consumers that are not idempotent. Every crash between forward and ack becomes a second charge or a second email.
• Store-and-forward for disposable data. Metric samples through a durable broker cost storage and latency to protect what the next sample replaces.
• Losing the trace context. A stored message must carry its trace id in metadata, or the consumer's work starts a new trace; see trace id vs span id.

When it shows up in interviews

Inside any design with a notification pipeline, an order flow or analytics events: "what happens if the email service is down?" or "how do you guarantee the event is published after the database commit?". The expected answers are store-and-forward through a queue, the outbox pattern for the second, and idempotent consumers for both. The HTTP side of retries is in REST API idempotency, and the lesson is message queues.

How to say it in an interview

"I split messages by what losing one costs. Telemetry is fire-and-forget: send without waiting, accept some loss, keep the sender fast. Anything with business meaning is store-and-forward: the broker writes it durably before acking the producer and delivers when the consumer is up. When the message must match a database change, I use an outbox table written in the same transaction, with a relay that publishes and marks rows sent. Both give at-least-once delivery, because a crash between delivery and ack means a resend, so consumers dedupe on a message id."

Frequently asked questions

What is the difference between store-and-forward and fire-and-forget?

Fire-and-forget sends a message without waiting for an ack or keeping a copy, so anything sent while the receiver is down is lost. Store-and-forward writes the message durably at an intermediary before acking the sender, then delivers it once the receiver is back, retrying until it is acknowledged.

When should I use fire-and-forget messaging?

When losing a message is cheaper than waiting for one: metric samples, debug logs, heartbeats, cache-warming hints. The next message replaces the lost one, and the sender should not slow down for an ack. Never for orders, payments or anything a user would notice missing.

Does store-and-forward guarantee exactly-once delivery?

No. It guarantees at-least-once: if the receiver processes a message and the ack is lost, the stored copy is sent again. Exactly-once effects come from the consumer, which records each message id it has applied and ignores a second copy.