SQS Standard vs FIFO: Ordering, Deduplication and Throughput

SQS Standard vs FIFO in one line: a standard queue delivers every message at least once, in roughly the order sent, at nearly unlimited throughput; a FIFO queue keeps strict order within each message group and does not introduce duplicates within a five-minute deduplication interval, at a much lower default throughput. Pick standard when your consumer can absorb duplicates and reordering, FIFO when order or no-duplicates is part of correctness. Every behaviour and number below comes from the Amazon SQS Developer Guide pages listed at the end, as of October 2026.

Side by side:

	Standard	FIFO
Delivery	at least once; a copy can arrive twice	exactly-once processing: no duplicates introduced within the 5-minute deduplication interval
Order	best-effort	strict within each MessageGroupId
Throughput	nearly unlimited API calls per second, per action	300 API calls per second per action, 3,000 messages per second with batches of 10; high throughput mode goes further
Name	any valid name	must end in .fifo

The problem it solves

Whether a queue is the right shape at all is covered in SQS vs SNS vs EventBridge. Once you have one, two questions remain:

• Can the consumer handle the same message twice? Charging a card twice is not acceptable; recomputing a thumbnail is.
• Does the order matter? "Order paid" processed before "order created" breaks a state machine; two independent uploads do not care.

Standard queues answer "no guarantees, maximum scale". FIFO queues answer "guarantees, per group, within limits".

The intuition

Standard: a highly distributed queue that tries hard. Because of its distributed design, more than one copy of a message might be delivered, and messages may occasionally arrive out of order, although SQS makes a best-effort attempt to keep the order. The consumer must be idempotent, as the message queues article explains.

FIFO: ordered lanes with a memory for duplicates. Three ideas define it:

• Message groups are the unit of order. Every message needs a MessageGroupId; within a group, messages are sent and received in strict order, and different groups may be processed out of order relative to each other. While messages from a group are in flight, SQS returns no more messages from that group until they are deleted or become visible again. One group is serial; parallelism comes from many groups.
• Deduplication has a window. A send with a MessageDeduplicationId already accepted in the last five minutes is acknowledged but not delivered. With content-based deduplication, SQS generates the ID as a SHA-256 hash of the body, not the attributes. A producer can therefore retry a failed send safely, as long as it receives an acknowledgment before the interval expires.
• Throughput is counted in API calls. Without high throughput mode, each partition handles 300 transactions per second per API action; with batches of 10, that is 3,000 messages per second. High throughput mode, which sets deduplication scope and the throughput limit to the message group, raises the limit by Region.

A standard queue cannot be converted into a FIFO queue; switching means a new .fifo queue.

Watch it run

The animation follows one order through a fan-out: an SNS topic pushes a copy into a billing queue and a shipping queue. Shipping is down, so its copy waits. Billing receives the order twice, standard at-least-once delivery, and its idempotent consumer skips the duplicate. A malformed message ends in the dead-letter queue, and the last frames swap the topic for an EventBridge bus. That duplicate is the problem this article is about.

The code

A FIFO queue and one send, illustrative. The URL holds a placeholder account ID; for high throughput mode, add DeduplicationScope=messageGroup,FifoThroughputLimit=perMessageGroupId to the attributes:

aws sqs create-queue --queue-name orders.fifo \
  --attributes FifoQueue=true,ContentBasedDeduplication=false
aws sqs send-message \
  --queue-url https://sqs.us-east-1.amazonaws.com/123456789012/orders.fifo \
  --message-body '{"order": 9, "event": "paid"}' \
  --message-group-id cust-1 \
  --message-deduplication-id order-9-paid

Next, a toy model, not SQS, built on the lesson's queue: the same order events for two customers go through a standard queue and a FIFO queue, with one producer retry. The standard queue swaps neighbours and redelivers copies at made-up rates; the point is that the rates are not zero. The FIFO queue drops a deduplication ID it has seen in the window and keeps each group in order:

import hashlib
import random
from collections import deque

sends = [  # (second, MessageGroupId, body, MessageDeduplicationId)
    (0, "cust-1", "c1 created", "e1"),
    (1, "cust-2", "c2 created", "e2"),
    (2, "cust-1", "c1 paid", "e3"),
    (3, "cust-1", "c1 paid", "e3"),      # the producer retries a timed-out send
    (4, "cust-2", "c2 paid", "e4"),
    (5, "cust-1", "c1 shipped", "e5"),
    (6, "cust-2", "c2 shipped", "e6"),
]

def standard_queue(sends, seed):
    """Toy model, not SQS: at-least-once, best-effort order. The retry is a
    second message, neighbours may swap, and a copy may arrive twice."""
    rng = random.Random(seed)
    stored = [body for _, _, body, _ in sends]
    for i in range(len(stored) - 1):
        if rng.random() < 0.3:
            stored[i], stored[i + 1] = stored[i + 1], stored[i]
    seen = []
    for body in stored:
        seen.append(body)
        if rng.random() < 0.2:
            seen.append(body)            # delivered again
    return seen

def fifo_queue(sends, window=300, content_based=False):
    """Toy model, not SQS: a send whose deduplication ID was accepted in the
    last `window` seconds is acknowledged but not delivered; order is kept
    per message group."""
    accepted, groups = {}, {}
    for t, group, body, dedup_id in sends:
        key = hashlib.sha256(body.encode()).hexdigest() if content_based else dedup_id
        if key in accepted and t - accepted[key] < window:
            continue                     # a duplicate inside the window
        accepted[key] = t
        groups.setdefault(group, deque()).append(body)
    seen = []
    while any(groups.values()):          # groups interleave, each stays in order
        for q in groups.values():
            if q:
                seen.append(q.popleft())
    return seen

STAGE = {"created": 0, "paid": 1, "shipped": 2}

def report(seen):
    dupes = len(seen) - len(set(seen))
    out_of_order = 0
    for cust in ("c1", "c2"):
        first = [b for i, b in enumerate(seen) if b.startswith(cust) and b not in seen[:i]]
        stages = [STAGE[b.split()[1]] for b in first]
        out_of_order += stages != sorted(stages)
    return f"duplicates={dupes} customers out of order={out_of_order}"

s = standard_queue(sends, seed=4)
print(s)
# ['c2 created', 'c1 paid', 'c1 created', 'c2 paid', 'c1 shipped', 'c1 paid',
#  'c2 shipped', 'c2 shipped']
print(report(s))                         # duplicates=2 customers out of order=1
f = fifo_queue(sends)
print(f)
# ['c1 created', 'c2 created', 'c1 paid', 'c2 paid', 'c1 shipped', 'c2 shipped']
print(report(f))                         # duplicates=0 customers out of order=0

The standard consumer saw the retried "c1 paid" twice, a "c2 shipped" redelivered, and customer 1 paid before created. The FIFO consumer saw each event once and each customer in order, while the two customers interleaved. Across many runs, and at the edges of the FIFO guarantees:

std = [standard_queue(sends, seed) for seed in range(1000)]
print(sum(len(r) - len(set(r)) for r in std) / 1000)  # 2.45 duplicates per run
print(sum("out of order=0" not in report(r) for r in std), "of 1000 runs reordered a customer")
# 163 of 1000 runs reordered a customer

late = [(0, "g", "refund 7", "r7"), (400, "g", "refund 7", "r7")]
print(fifo_queue(late))                       # ['refund 7', 'refund 7']  retry after the window
twins = [(0, "g", "top up 10", "t1"), (30, "g", "top up 10", "t2")]
print(fifo_queue(twins, content_based=True))  # ['top up 10']  a real second top-up dropped
print(fifo_queue(twins))                      # ['top up 10', 'top up 10']

def drain_ticks(messages, consumers):
    """Toy model of message groups: a group with a message in flight
    gives out no other message, so one group is processed serially."""
    queue, ticks = deque(messages), 0
    while queue:
        busy, held = set(), deque()
        while queue and len(busy) < consumers:
            group = queue.popleft()
            if group in busy:
                held.append(group)            # its group is locked this tick
            else:
                busy.add(group)
        queue = held + queue
        ticks += 1
    return ticks

for groups in (1, 3, 12):
    msgs = [f"g{i % groups}" for i in range(12)]
    print(groups, "groups, 3 consumers:", drain_ticks(msgs, 3), "ticks")
# 1 groups, 3 consumers: 12 ticks
# 3 groups, 3 consumers: 4 ticks
# 12 groups, 3 consumers: 4 ticks

A retry after the window is a new message. Content-based deduplication drops a real second top-up with the same body; an explicit ID per event keeps it. And three consumers drain one group no faster than one: order costs parallelism unless the group ID is fine-grained.

The complexity

The trade is scale for guarantees:

• Standard scales without design work, and moves the cost to the consumer: idempotency keys and tolerance for reordering.
• FIFO costs a ceiling per API action, unless high throughput mode is on, and a design decision: the group ID decides both what stays ordered and how many consumers can work at once.
• Batching matters more on FIFO: the limit counts API calls, and a batch of up to 10 is one call.

Where it goes wrong

• One group ID for everything. The whole queue becomes one serial lane. Use the narrowest key that still needs order.
• Content-based deduplication for events that can repeat. Two genuine top-ups with the same body become one. Send an explicit deduplication ID per event.
• Treating "exactly-once processing" as "my code runs once". A message that is received but not deleted before the visibility timeout becomes available again. Keep side effects idempotent anyway.
• A long-running message blocking its group. Nothing else from that group is delivered while it is in flight.

When it shows up in interviews

It appears as "how do you guarantee orders are processed in sequence?" or "how do you avoid double charges?" in payment, ledger and inventory designs. A strong answer chooses FIFO only where order is a requirement, names the group key, and still keeps the consumer idempotent. When the question becomes replay or many consumers reading the same history, it is a different tool, covered in Kinesis vs SQS vs MSK.

How to say it in an interview

"Standard SQS is at-least-once with best-effort order and nearly unlimited throughput, so the consumer must be idempotent and tolerate reordering. FIFO gives strict order per message group and doesn't introduce duplicates within a five-minute deduplication window, keyed by a deduplication ID or a SHA-256 of the body. Its limit is per API action, higher with batching and high throughput mode. Message groups are the unit of both order and parallelism, so I'd key them by customer or order, not one global group. I'd still make the consumer idempotent, because an undeleted message comes back after the visibility timeout."

The queue model, dead-letter queues and fan-out are in SQS vs SNS vs EventBridge.

Frequently asked questions

What is the difference between SQS standard and FIFO queues?

Standard queues deliver at least once, in best-effort order, with nearly unlimited API calls per second. FIFO queues keep strict order within each message group and don't introduce duplicates within the five-minute deduplication interval, but have a throughput limit per API action. FIFO queue names must end in .fifo.

Does an SQS FIFO queue guarantee exactly-once delivery?

It guarantees that retried sends within the five-minute deduplication interval don't create duplicates, and a delivered message stays unavailable until it is deleted. If the consumer does not delete it before the visibility timeout, it is received again. Keep the consumer idempotent.

How do I increase SQS FIFO throughput?

Batch up to 10 messages per call, because the limit counts API calls. Enable high throughput mode, which sets deduplication scope and the throughput limit to the message group. Spread messages across many message group IDs, since each group is processed in order, one batch at a time.

Sources

• Amazon SQS queue types — AWS Documentation
• Amazon SQS standard queues — AWS Documentation
• Amazon SQS FIFO queues — AWS Documentation
• FIFO queue delivery logic in Amazon SQS — AWS Documentation
• Exactly-once processing in Amazon SQS — AWS Documentation
• Using the message deduplication ID in Amazon SQS — AWS Documentation
• High throughput for FIFO queues in Amazon SQS — AWS Documentation
• Amazon SQS message quotas — AWS Documentation
• Moving from a standard queue to a FIFO queue — AWS Documentation