40 System Design Interview Questions, Answered with Trade-offs

A system design round is not a quiz on components. It is a conversation about trade-offs: you propose a shape, the interviewer pushes on it, and what they listen for is whether you know what each piece costs and what breaks first. These forty questions start with how to open the conversation and end with the classic cases, from a URL shortener to a collaborative editor.

Every answer is short on purpose: something you can say out loud in under a minute, with the cost stated next to the benefit. Each one links the lesson that animates it, and the cases link the longer write-ups where they exist. Numbers in the case answers are the lessons' own working assumptions, not measurements of any real product, and the three mechanisms that are easiest to get wrong are run as a small toy model at the end.

How to use this list

Answer each question aloud before reading ours. If your answer names a component but not what it costs, it is half an answer: the follow-up is always "and what does that break?". A cache that is never stale, a queue that never duplicates and a database that scales writes for free do not exist, and saying so first is what sounds senior.

Fundamentals

1. What is a system design interview actually testing?

Whether you can choose a shape for a system and defend it. Latency, cost, correctness and uptime pull against each other, so there is no correct answer, only a defended one: say which constraint you protect when they collide and which one you let give. A single server with one database beats a clever distributed design until it genuinely cannot cope, so design for the load you have plus the next order of magnitude, no further.

Lesson: What is system design

2. How do you structure your answer?

Agree the requirements and the numbers before drawing a box: reads and writes per second, data size, the latency you must hit. Then sketch the components, trace one request end to end, and name the first bottleneck. Fix only that one. Every piece added after it is one more thing to deploy, monitor and fund, so each needs a reason you can say out loud.

Lesson: What is system design

3. How do you do back-of-the-envelope estimates?

Multiply rate by size and round. 2 000 reads a second at 8 KB each is 16 MB/s. 20 000 viewers at 5 Mb/s is 100 Gb/s of egress, which decides a video design on its own. A day has 86 400 seconds, so five million jobs a day is about 58 a second on average, and the average is not the target when every hour boundary brings a spike. Then compare reads with writes: a read-heavy load points at caches and replicas, a write-heavy one at sharding, because replicas and caches add no write capacity.

Lesson: What is system design · Cases: video streaming, job scheduler

4. What makes a REST API well designed?

The path names a resource, the HTTP method is the verb, and the status code carries the outcome. POST /articles answers 201 Created with a Location header, GET /articles/91 answers 200, DELETE answers 204, and the next GET is a 404. GET is safe and cacheable; PUT and DELETE are idempotent. /getUserById?id=7 puts the verb in the path. Some operations, a refund or a publish, are verbs first; choose a convention, version it, and hold the line.

Lesson: Designing a REST API

5. What is idempotency, and how do you make a POST safe to retry?

An operation is idempotent when doing it twice leaves the same state as doing it once. PUT and DELETE are defined that way; POST is not, so a client that retries after a timeout can create two orders. Have the client send an idempotency key; the server stores the key with the result and returns the stored result on a repeat. The same key is what stops an at-least-once consumer from acting twice.

Lesson: Designing a REST API · Case: notification system · Article: design a notification system

6. Polling, WebSockets or server-sent events?

Polling asks "anything new?" on a timer and mostly wastes the request. A WebSocket starts as an HTTP request asking to upgrade, the server answers 101 Switching Protocols, and then either side can write at any moment. The cost is that every open socket is state on one particular server, so you need sticky routing or a shared pub/sub layer to fan messages out. For updates every few seconds, plain polling or server-sent events, a one-way stream from server to client over ordinary HTTP, are far less to operate.

Lesson: WebSockets and realtime

7. Metrics, logs and traces: what is each for?

Metrics are cheap numbers over time, such as request rate, error rate and latency percentiles, and they are what you alert on. Logs record individual events with context and get expensive at volume. A trace follows one request across every service it touched, the only view that finds the slow hop. Alert on percentiles, not averages, because an average hides the slow tail, and keep metric labels low-cardinality: a label per user ID multiplies the series you pay for.

Lesson: Observability basics

8. How does distributed tracing work, and why sample?

A trace is a tree of spans, each with a name, a start, a duration and a parent. A trace ID and the current span ID travel in the request headers, which is the only reason hops in separate services land in one picture. Recording every request is expensive, so you sample: head sampling decides at the first hop and is cheap, but discards the rare slow request you needed; tail sampling keeps the interesting traces but must buffer every span until the trace ends.

Lesson: Tracing a request

Scaling

9. Vertical or horizontal scaling?

Vertical scaling buys a bigger machine: no code changes, but the ceiling is the largest box on the market, resizing usually costs downtime, and it is still one machine that can fail. Horizontal scaling adds machines: no ceiling, but it only works when any machine can answer any request. A second server does not make one request faster; it adds redundancy and total capacity. Scale out once the single machine is actually full.

Lesson: Vertical vs horizontal scaling

10. What does "stateless" mean, and where does the state go?

A stateless server keeps no request-specific state locally, so any server can answer any request and adding servers adds capacity. Sessions, carts and uploads move into a shared store: a database, a cache, object storage. The alternative, sticky sessions, pins users to one server, which unbalances the pool and logs everyone on it out when it dies.

Lessons: Vertical vs horizontal scaling, Load balancing

11. How does a load balancer choose a server?

Round-robin cycles through the pool and is fine when every request costs the same. Least-connections favours whoever is least busy, which wins when request cost varies widely. A hash on a key, such as the client, always sends the same client to the same server, at the cost of an even spread. Health checks quietly remove servers that stop responding.

Lesson: Load balancing · Article: round robin vs least connections

12. Isn't the load balancer a single point of failure?

It is if there is only one. Everything depends on it, so it needs its own redundancy: at least two, with failover between them, or a managed balancer that is already redundant. Ask the same question of every shared piece you add, whether it is the counter store behind a rate limiter, the session store or the queue.

Lesson: Load balancing

13. Which rate limiting algorithm would you use?

A token bucket refills at a steady rate up to a maximum, so a client can burst up to the bucket size and then settles to the refill rate. A fixed window is the cheapest to build and the easiest to abuse: a client can spend a full budget on each side of the boundary, twice the limit in a moment. Sliding windows smooth that out for more memory or arithmetic. Reject with 429 Too Many Requests and a Retry-After header. The code below runs a token bucket.

Lesson: Rate limiting · Article: rate limiting algorithms compared

14. Where does a distributed rate limiter keep its counter?

Per-node counters need no network hop, but across three gateway nodes a caller can spend up to three times the limit, because each node grants a full budget. One shared counter store is exact, and it adds a round trip to every request plus a hard dependency. Decide in advance what happens when that store is unreachable: fail open and serve everything unprotected, or fail closed and reject everyone.

Case: Design a rate limiter

15. What does a CDN do, and what should it not cache?

It keeps copies of your static files on servers spread around the world and routes each visitor to a nearby one, so bytes travel a shorter way and the origin only sees the misses. Do not cache a page personalised for one user: nothing about it is reusable, and a shared cache could hand it to the wrong person, so mark it private. A bad deploy is now cached at every edge and purging takes minutes; versioned filenames sidestep that, because a fix is a new object.

Lesson: Content delivery networks

Caching

16. When does a cache help, and how do you know it is working?

When a small set of items is requested far more often than everything else. If almost every request asks for something different, every lookup is still a miss, and the cache buys only memory cost and a stale-data bug. Measure the hit ratio before believing in it: ninety hits in a hundred is a cache, nine is a liability.

Lesson: Caching · Article: caching explained

17. Walk through cache-aside.

On a read, look the key up in the cache; a hit answers immediately. A miss reads the database, stores the result in the cache with a TTL, and returns it. On a write, update the database, then delete the cached key so the next read fetches the new value. The application owns this logic, and the cache only ever holds what was asked for. Write-through, which writes the cache on every database write, keeps recently written keys warm at the cost of caching things nobody reads.

Lesson: Caching · Article: caching explained

18. TTL or invalidate on write?

Invalidation asks when a cached copy stops being true. A TTL is cheap and bounds how long a wrong copy can survive, but it does not prevent one. Deleting the key the moment the source changes is exact, and it couples the writer to every cache that has ever held that key. The usual answer is both: delete on write, with a TTL as the safety net for the delete that got lost.

Lesson: Invalidation and eviction

19. LRU or LFU eviction?

Eviction decides what to drop when the cache is full. LRU drops the key untouched for longest, betting that what nobody asked for lately will not be asked for soon. LFU drops the least often requested, which keeps steady favourites through a burst of one-off reads but has to track counts. Both run in O(1): LRU with a hash map and a doubly linked list, LFU with frequency buckets.

Lesson: Invalidation and eviction · Articles: LRU cache from scratch, LFU vs LRU

20. What is a cache stampede, and how do you prevent it?

A popular key expires under heavy load, every waiting request misses at the same instant, and all of them go to the database together. Let one request refill the key while the others wait for it, add random jitter to TTLs so keys do not expire in step, and refresh the hottest keys before they expire.

Lesson: Invalidation and eviction

21. How do you spread a cache over many machines?

Consistent hashing. Keys and machines are hashed onto a ring, and each key belongs to the first machine clockwise, so a machine joining or leaving moves only its own arc. With key modulo machine count, adding one machine remaps almost every key and every machine starts cold; the code below measures 80% of keys moving under modulo against about a fifth on a ring. Virtual nodes, each machine scattered as many points, even out the share each one owns.

Case: Design a distributed cache · Article: consistent hashing explained

Databases

22. SQL or NoSQL?

A relational database enforces a schema, stores each fact once and joins tables at query time with transactional guarantees. A document store keeps whole objects together, lets fields vary between records, and expects you to duplicate data so a read needs no join, which means every copy must be updated on write. "Schemaless" means the schema is enforced by your application instead of the database. Reach for documents when a shape genuinely varies and is read whole; keep anything that must reconcile, like invoices, in tables.

Lesson: SQL vs NoSQL · Article: SQL vs NoSQL: how to choose

23. What does replication buy you, and what does it not?

Writes land on the primary, which streams its changes to replicas, and reads can be served from any copy. That multiplies read capacity and gives you a standby to promote when the primary dies. It does not multiply write capacity: every write still funnels through the one primary.

Lesson: Database replication · Article: replication lag explained

24. What is replication lag, and how do you hide it from users?

With asynchronous replication a replica is behind by however long the changes take to arrive, so a user can save a comment, refresh, and read the old page from a replica. Read-your-writes fixes the part users notice: for a while after a user writes, send that user's reads to the primary or pin the session to a replica that already has the write. Decide up front which reads are allowed to be stale.

Lessons: Database replication, Consistency models

25. Synchronous or asynchronous replication?

Synchronous replication makes a write wait until a replica confirms it, so a user no longer reads their own write and sees the old value, and every write pays that round trip. Asynchronous replication is fast and leaves that window open, and a failover while a replica is behind can lose committed transactions. A common middle ground is one synchronous standby with the rest asynchronous.

Lesson: Database replication

26. What is sharding, and how do you pick a shard key?

Sharding splits one dataset across several databases by a shard key; each shard owns a slice and serves its own reads and writes, so write capacity finally scales out. A good key appears in most queries, so a query touches one shard, and spreads load evenly. A query without the key is sent to every shard and the results are merged. A key such as "newest timestamp" sends every write to the same shard.

Lesson: Database sharding · Article: shard keys and hot shards

27. What gets hard after sharding, and when should you shard?

Cross-shard joins, unique constraints and transactions get hard or vanish, a poorly chosen key leaves one hot shard doing the work while the rest idle, and resharding a live system is a migration project. So shard only after indexes, caching and read replicas are genuinely exhausted, and when you do, choose a placement scheme that moves little data when shards are added.

Lesson: Database sharding

Messaging

28. Why put a queue between two services?

The producer writes a message and returns immediately, and workers pull messages when they have capacity. Bursts pile up in the queue instead of timing out, the backlog is visible, and the two sides can be deployed, scaled and restarted independently. If the caller genuinely needs the answer now, a queue only hides the waiting somewhere less visible.

Lesson: Message queues · Article: message queues explained

29. At-most-once, at-least-once or exactly-once?

At-most-once may lose a message. At-least-once loses none but can deliver the same message twice, so consumers must be idempotent: processing a message twice has to be harmless, typically by recording the IDs already handled or by making the write an upsert. Treat exactly-once as something you build from at-least-once delivery plus idempotent consumers, not something to assume.

Lesson: Message queues · Case: ad click aggregation

30. What happens to a message that keeps failing?

Retry with exponential backoff, so a struggling dependency is not hammered at full speed; the job scheduler case sets run_at = now() + 2^attempts after each failure. Once the attempts run out, five in that case, move the message to a dead-letter queue, so one poison payload cannot block the work behind it, and alert on that queue's depth.

Lesson: Message queues · Case: Design a job scheduler

31. Can a queue keep messages in order?

Within one queue or partition read by one consumer, yes; across parallel consumers, no, which is one of the costs the lesson lists. When order matters per entity, partition by that entity's key, so every event for one account lands on the same partition and the same worker, and promise order only per key. Global order means a single consumer, and that caps throughput.

Lesson: Message queues

Consistency

32. What does the CAP theorem say?

When a network partition cuts your nodes apart, a node that cannot reach its peers must either refuse, staying consistent and becoming unavailable, or answer from what it knows, staying available and risking conflicting data. Partitions are not optional, so the real decision is what a node does during one. With every node reachable you can have both. Moving money wants the consistent side; a like count is fine on the available one.

Lesson: Consistency and CAP · Article: CAP theorem explained

33. Strong, eventual, read-your-writes or causal consistency?

Strong consistency always shows the newest committed write, and costs coordination with the leader or a quorum on every read, plus stalls during a failover. Eventual consistency may show any recent copy; it is cheap and stays up, and orders nothing. Between them sit the guarantees users notice: read-your-writes, where you never see a version older than your own last write, and causal order, where a reply never appears before the message it answers. Choose the weakest model whose anomalies a user would never notice.

Lesson: Consistency models

34. How do quorum reads and writes work?

With N replicas, a write waits for W acknowledgements and a read consults R replicas. If R + W is greater than N, every read set shares at least one replica with every write set, so the read sees the newest value; take the higher version and repair the stale replica in the background. N = 3, W = 2, R = 2 is the classic setting. Raising W buys durability and costs write availability: with W = 3, one unreachable replica stops all writes. The code checks every combination for N = 3.

Case: Design a key-value store

35. How do you stop two buyers getting the last unit, or two guests the same room?

Make the check and the write one atomic step. A conditional decrement, UPDATE stock SET available = available - 1 WHERE sku = ? AND available >= 1, where zero rows updated means sold out; reading the count and then writing it leaves a gap another cart fits straight into. For a hotel stay, write every night in one transaction and keep a unique constraint on room and night as the last defence. The cost is contention on the hottest row, and a hold with an expiry returns stock from abandoned carts.

Cases: e-commerce inventory, hotel booking

Case studies

36. Design a URL shortener: what are the key decisions?

It is read-heavy, 500 redirects a second against 5 writes in the lesson's numbers, with one access pattern, key in and long URL out, so a key-value store with a cache in front fits. A base62 counter gives the shortest collision-free keys, and leaks how many links exist through one shared sequence; random keys hide that and need a uniqueness check on every write. Redirect with 302: a 301 is cached by the browser, later clicks never reach you, and the click counts stop.

Case: Design a URL shortener · Article: URL shortener interview walkthrough

37. Design a chat app: what is the hard part?

Finding the socket. Every phone holds an open connection to one of many gateways, so a session registry maps each user to the gateway holding their socket, and it goes stale the moment a phone drops off a train. Store the message with a sequence number before acknowledging the sender: that write on the hot path is the only reason delivery survives a crashed gateway. A recipient with no socket gets a push notification, and the phone pulls from the stored sequence when it reconnects.

Case: Design a chat app · Article: chat app interview walkthrough

38. News feed: fan-out on write or on read?

Fan-out on write assembles each follower's feed when a post is published, so opening the app is one cheap read, and a celebrity's single post becomes tens of millions of inbox writes. Fan-out on read publishes instantly and makes every open merge many timelines. Real systems split it: write for ordinary accounts, read for the few enormous ones.

Case: Design a news feed · Article: fan-out on write vs read

39. Design a notification system: where can it go wrong?

One event must become the right messages on the right channels, push, email or SMS, honouring each user's preferences. A queue between the event and the senders absorbs bursts, 200 000 a minute in the lesson's numbers, and turns a provider's outage into queue depth rather than lost notifications; you drain it by adding workers. Because the queue delivers at least once, every notification carries an idempotency key so a retry cannot send it twice.

Case: Design a notification system · Article: notification system walkthrough

40. What one idea unlocks each of the other classic cases?

• Search autocomplete: a trie whose nodes store their own top-10, rebuilt offline, so a prefix is one lookup; suggestions lag until the next rebuild.
• File storage: split files into chunks, hash each, upload only the unknown ones; metadata in a database, bytes in blob storage.
• Video streaming: several bitrate renditions cut into segments with a manifest, served from a CDN, so the player steps down instead of buffering.
• Ride matching: driver positions in memory, indexed by grid cell; read the neighbouring cells too, and rank by time of arrival, not distance.
• Geo proximity search: a geohash turns nearness into a prefix match on a plain index; a quadtree when density varies wildly.
• Payment ledger: append-only double entry, debit and credit in one transaction, periodic balance snapshots.
• Job scheduler: a lease with a heartbeat, handlers safe to run twice, backoff, and a dead-letter queue.
• Ad click aggregation: an ID on every event, counts per campaign per minute, and a watermark that decides when a minute is closed.
• Web crawler: a frontier partitioned by host for politeness, a URL hash set, and a content fingerprint for mirrors.
• Leaderboard: a score-ordered set in memory, so the top hundred is a range read; approximate deep ranks from score buckets.
• Collaborative editor: apply each keystroke locally first, then transform position-based edits against each other or merge by character identity.

Watch it run

Questions 1 to 3 in one animation. It opens on the rule that system design picks the shape of a system before the traffic picks one for you, then agrees the numbers before any box is drawn: 2 000 reads a second, 40 writes a second, 8 KB each. One request travels from the client to the API and on to the database, which answers in 38 ms, the slowest hop on the path, and the answer returns the way it came. Then reads climb to 9 000 a second and the database saturates first, so that is the bottleneck to fix, and only that one: a cache in front of the database absorbs the reads. The last frames make the cost visible, because the cache is one more thing to deploy, monitor and fund, and close on the point of the whole list: there is no correct design, only a defended one.

The code

A toy model, not a production cache, database or gateway: three of the answers above reduced to a few lines each so the numbers can be checked. Question 21's remapping, measured over 10 000 keys when a fifth machine joins four; question 34's overlap rule, checked against every read and write set for three replicas; and question 13's token bucket on a fake clock. The hash is MD5 from hashlib, so the output is the same on every run:

import bisect
import hashlib
from itertools import combinations

def h(key):
    """A stable hash (Python's hash() of a str changes between runs)."""
    return int(hashlib.md5(key.encode()).hexdigest(), 16)

KEYS = [f"user:{i}" for i in range(10_000)]

# Q21: how many keys change owner when a 5th cache node joins 4?
def modulo(nodes):
    return {k: nodes[h(k) % len(nodes)] for k in KEYS}

def ring(nodes, vnodes=100):
    points = sorted((h(f"{n}#{v}"), n) for n in nodes for v in range(vnodes))
    hashes = [p for p, _ in points]
    def owner(k):
        i = bisect.bisect(hashes, h(k)) % len(points)   # first point clockwise
        return points[i][1]
    return {k: owner(k) for k in KEYS}

def moved(before, after):
    return sum(before[k] != after[k] for k in KEYS) / len(KEYS)

four, five = ["a", "b", "c", "d"], ["a", "b", "c", "d", "e"]
print(f"modulo: {moved(modulo(four), modulo(five)):.0%} of keys moved")   # modulo: 80% of keys moved
print(f"ring:   {moved(ring(four), ring(five)):.0%} of keys moved")       # ring:   21% of keys moved

# Q34: with N = 3 replicas, does every read set meet every write set?
def always_overlap(n, r, w):
    nodes = range(n)
    return all(set(rs) & set(ws)
               for rs in combinations(nodes, r) for ws in combinations(nodes, w))

for r, w in [(1, 1), (1, 2), (2, 2), (1, 3)]:
    print(f"N=3 R={r} W={w}  R+W>N: {r + w > 3}  overlap: {always_overlap(3, r, w)}")
# N=3 R=1 W=1  R+W>N: False  overlap: False
# N=3 R=1 W=2  R+W>N: False  overlap: False
# N=3 R=2 W=2  R+W>N: True  overlap: True
# N=3 R=1 W=3  R+W>N: True  overlap: True

# Q13: a token bucket of 5 tokens refilling at 1 per second, on a fake clock
class TokenBucket:
    """Toy model: one bucket, a fake clock, no concurrency."""
    def __init__(self, capacity, rate):
        self.capacity, self.rate = capacity, rate
        self.tokens, self.last = capacity, 0.0
    def allow(self, now):
        self.tokens = min(self.capacity, self.tokens + (now - self.last) * self.rate)
        self.last = now
        if self.tokens >= 1:
            self.tokens -= 1
            return 200
        return 429

bucket = TokenBucket(capacity=5, rate=1.0)
print([bucket.allow(0.0) for _ in range(8)])   # [200, 200, 200, 200, 200, 429, 429, 429]
print([bucket.allow(2.0) for _ in range(3)])   # [200, 200, 429]

The modulo figure is not bad luck. Going from four machines to five, a key keeps its owner only when its hash gives the same remainder for 4 and 5, which is 4 values out of every 20, so four keys in five move. On the ring the new machine takes roughly its fair share, a fifth, and nothing else moves. The bucket shows the burst: five requests pass at once, the rest get 429, and two seconds later two tokens have refilled.

How to say it in an interview

Numbers, one request, first bottleneck, cost. "At 2 000 reads and 40 writes a second this is read-heavy, so one primary with read replicas and a cache-aside layer in front is enough. I trace a read through the load balancer to a stateless API server and the cache; the first thing to saturate is the database, and the cache absorbs that. The cost is staleness, so writes delete the key and every entry has a TTL. If writes grow past one primary, I shard by a key that every query carries." Each sentence names a mechanism and what it costs, which is the whole interview.

Sources

• RFC 9110: HTTP Semantics — safe and idempotent methods, status codes, Retry-After
• RFC 6585: Additional HTTP Status Codes — 429 Too Many Requests
• RFC 6455: The WebSocket Protocol — the opening handshake and 101 Switching Protocols