System design interview questions: a framework, trade-offs and real code
How to structure a system design answer, the building blocks interviewers expect, and small runnable examples of the classic techniques.
System design interviews ask you to design a service (a URL shortener, a news feed, a chat app) in about 45 minutes. Interviewers look for a clear process: clarify requirements, estimate scale, sketch the high-level design, go deep on the hardest parts and discuss trade-offs. The building blocks are load balancers, caches, databases (replication and sharding), queues, CDNs and consistency choices. Below are 18 questions; the code examples were run and show real output.
Basic system design interview questions
Basic1. How should you structure a system design answer?
Clarify functional and non-functional requirements (scale, latency, consistency, availability), estimate traffic and storage, define the API and data model, draw the high-level components, then go deep on the riskiest part (for example the database or the fan-out) and finish with bottlenecks, failure modes and trade-offs. Talk through your reasoning the whole time.
Basic2. How do you do back-of-the-envelope estimates?
Start from users and actions per user, convert to requests per second (a day has about 86,400 seconds), allow a peak factor, and multiply by object size for storage and bandwidth. Round aggressively: the goal is the order of magnitude, which decides whether one database is enough.
Pythondau = 10_000_000 # daily active users (assumption)
writes_per_user = 2
reads_per_write = 50
bytes_per_item = 500
write_qps = dau * writes_per_user / 86_400
print(f'write QPS ~{write_qps:,.0f}, peak (x3) ~{3 * write_qps:,.0f}')
print(f'read QPS ~{write_qps * reads_per_write:,.0f}')
print(f'new storage per year ~{dau * writes_per_user * bytes_per_item * 365 / 1e12:.1f} TB')Output (Python 3.12.10)write QPS ~231, peak (x3) ~694 read QPS ~11,574 new storage per year ~3.6 TB
Basic3. What does a load balancer do?
It spreads requests across healthy servers, removes failed ones using health checks, and lets you add capacity behind one address. Layer 4 balancers route by IP and port; layer 7 balancers understand HTTP and can route by path or host, terminate TLS and apply rules. Keep servers stateless so any server can handle any request.
Basic4. When and how do you add a cache?
Add a cache when the same data is read far more often than it changes. In the cache-aside pattern the app reads from the cache, and on a miss reads the database and fills the cache with a TTL. You must decide how to invalidate entries on writes and what eviction policy to use, such as least recently used.
Pythonfrom collections import OrderedDict
class LRU:
def __init__(self, cap): self.cap, self.d = cap, OrderedDict()
def get(self, k):
if k not in self.d: return None
self.d.move_to_end(k); return self.d[k]
def put(self, k, v):
self.d[k] = v; self.d.move_to_end(k)
if len(self.d) > self.cap: print('evict', self.d.popitem(last=False)[0])
c = LRU(2)
c.put('a', 1); c.put('b', 2); c.get('a'); c.put('c', 3)
print('keys now', list(c.d))Output (Python 3.12.10)evict b keys now ['a', 'c']
Basic5. What is a CDN for?
A content delivery network caches static files (images, scripts, video) on servers close to users, cutting latency and taking load off your origin. Some CDNs also cache API responses or run code at the edge.
Basic6. SQL or NoSQL: how do you choose?
Relational databases give you joins, transactions and a flexible query language, and suit data with relationships and strong consistency needs. NoSQL stores (key-value, document, wide-column, graph) trade some of that for horizontal scale, flexible schemas or a data model that fits the access pattern. Choose from the queries you must serve, not from fashion.
Intermediate system design interview questions
Intermediate7. What is the difference between replication and sharding?
Replication copies the same data to several nodes: a leader takes writes and followers serve reads and take over on failure. Sharding splits data across nodes by a key, so each node holds a subset, which scales writes and storage. Large systems use both: every shard is replicated.
Intermediate8. What is consistent hashing?
Servers and keys are placed on a hash ring, and each key belongs to the next server clockwise. When a server is added or removed, only the keys next to it move, instead of nearly all keys as with hash modulo N. Virtual nodes smooth out the distribution.
Pythonimport bisect, hashlib
def h(s): return int(hashlib.md5(s.encode()).hexdigest(), 16)
def ring(nodes, vnodes=100):
return sorted((h(f'{n}#{i}'), n) for n in nodes for i in range(vnodes))
def owner(r, key):
i = bisect.bisect(r, (h(key),)) % len(r); return r[i][1]
keys = [f'user{i}' for i in range(10_000)]
r3, r4 = ring(['A', 'B', 'C']), ring(['A', 'B', 'C', 'D'])
moved = sum(owner(r3, k) != owner(r4, k) for k in keys)
mod_moved = sum(h(k) % 3 != h(k) % 4 for k in keys)
print(f'consistent hashing: {moved / len(keys):.1%} of keys moved')
print(f'hash mod N: {mod_moved / len(keys):.1%} of keys moved')Output (Python 3.12.10)consistent hashing: 23.9% of keys moved hash mod N: 74.8% of keys moved
Intermediate9. What does the CAP theorem say?
When a network partition happens, a distributed system must choose between consistency (every read sees the latest write) and availability (every request gets a response). Without a partition you can have both; the PACELC extension adds that even then you trade latency against consistency.
Intermediate10. How does rate limiting work?
A token bucket holds up to a fixed number of tokens and refills at a steady rate; each request spends a token and is rejected (HTTP 429) when the bucket is empty, which allows short bursts. Limits are usually kept per user or API key in a shared store such as Redis.
Pythonclass TokenBucket:
def __init__(self, capacity, rate):
self.capacity, self.rate, self.tokens, self.t = capacity, rate, capacity, 0.0
def allow(self, now):
self.tokens = min(self.capacity, self.tokens + (now - self.t) * self.rate); self.t = now
if self.tokens >= 1: self.tokens -= 1; return True
return False
b = TokenBucket(capacity=3, rate=1) # burst of 3, then 1 request per second
times = [0, 0.1, 0.2, 0.3, 0.4, 1.5, 1.6, 3.0]
print([('ok' if b.allow(t) else '429') for t in times])Output (Python 3.12.10)['ok', 'ok', 'ok', '429', '429', 'ok', '429', 'ok']
Intermediate11. Why use a message queue?
A queue decouples producers from consumers: work is accepted quickly and processed asynchronously, spikes are absorbed, and failed jobs can be retried. You must handle duplicates (at-least-once delivery), ordering where it matters, poison messages (a dead-letter queue) and back pressure.
Intermediate12. What is idempotency, and why does it matter?
An idempotent operation has the same effect whether it runs once or many times. Because networks fail and clients retry, APIs that change state (payments, orders) accept an idempotency key and return the stored result for a repeated key instead of acting twice.
Intermediate13. How do retries go wrong, and how do you fix them?
Immediate, synchronized retries from many clients can overload a struggling service. Use timeouts, a retry limit, exponential backoff with random jitter, and circuit breakers that stop calling a failing dependency for a while.
Pythonimport random
random.seed(0)
base, cap = 0.1, 5.0
for attempt in range(6):
backoff = min(cap, base * 2 ** attempt)
print(f'attempt {attempt}: wait up to {backoff:.1f}s, jittered wait {random.uniform(0, backoff):.2f}s')Output (Python 3.12.10)attempt 0: wait up to 0.1s, jittered wait 0.08s attempt 1: wait up to 0.2s, jittered wait 0.15s attempt 2: wait up to 0.4s, jittered wait 0.17s attempt 3: wait up to 0.8s, jittered wait 0.21s attempt 4: wait up to 1.6s, jittered wait 0.82s attempt 5: wait up to 3.2s, jittered wait 1.30s
Advanced system design interview questions
Advanced14. Design a URL shortener.
Write path: accept a long URL, generate a unique ID (from a counter or ID service, or a random value with a collision check), encode it in base62 to get a short code, and store code to URL in a key-value store. Read path: look up the code (cache hot codes), and redirect with 301 or 302 depending on whether you need analytics. Discuss custom aliases, expiry, abuse checks and analytics events sent to a queue.
Pythonimport string
ALPHABET = string.digits + string.ascii_lowercase + string.ascii_uppercase # 62 characters
def encode(n):
s = ''
while True:
n, r = divmod(n, 62); s = ALPHABET[r] + s
if n == 0: return s
def decode(s):
n = 0
for ch in s: n = n * 62 + ALPHABET.index(ch)
return n
for n in (125, 3_844, 1_000_000_007):
code = encode(n); print(n, '->', code, '->', decode(code))
print('7-character codes available:', f'{62 ** 7:,}')Output (Python 3.12.10)125 -> 21 -> 125 3844 -> 100 -> 3844 1000000007 -> 15FTGn -> 1000000007 7-character codes available: 3,521,614,606,208
Advanced15. Design a news feed: fan-out on write or on read?
Fan-out on write pushes each new post into every follower's precomputed feed, so reads are fast but writes are heavy for accounts with millions of followers. Fan-out on read assembles the feed at request time, so writes are cheap and reads are heavy. Large systems mix them: push for typical accounts, pull for very large ones, then rank and cache the result.
Advanced16. Design a chat system.
Clients keep a persistent connection (WebSockets) to a gateway tier; a presence service tracks who is online; messages go to a service that assigns a per-conversation sequence number, stores them durably and delivers them to online recipients through their gateway, with push notifications for offline users. Discuss ordering, delivery receipts, group fan-out, history storage and end-to-end encryption.
Advanced17. How do you generate unique IDs across many servers?
Options include database sequences (simple, a central bottleneck), UUIDs (no coordination, but long and unordered unless you use time-ordered versions such as UUIDv7), and Snowflake-style IDs that pack a timestamp, a machine ID and a per-machine sequence into 64 bits so IDs are unique and roughly time-ordered.
Pythondef snowflake(ms_since_epoch, machine, seq):
assert machine < 1024 and seq < 4096
return (ms_since_epoch << 22) | (machine << 12) | seq
a = snowflake(1_700_000_000_000, machine=7, seq=0)
b = snowflake(1_700_000_000_000, machine=7, seq=1)
c = snowflake(1_700_000_000_001, machine=3, seq=0)
print(a, b, c)
print('time-ordered:', a < b < c, ' bits used:', c.bit_length())Output (Python 3.12.10)7130316800000028672 7130316800000028673 7130316800004206592 time-ordered: True bits used: 63
Advanced18. How do you make a design reliable and observable?
Remove single points of failure with redundancy across zones, degrade gracefully (serve cached or partial results), protect dependencies with timeouts and circuit breakers, and plan capacity with headroom. Measure what users feel (latency percentiles, error rates, saturation) with metrics, logs and traces, and define SLOs with alerts.
How to practice with this page
- Practice out loud with a timer: requirements, estimates, design, deep dive, trade-offs.
- Draw the same design twice: once for 1,000 users and once for 100 million, and say what changed.
- For every component, know what happens when it fails.
- Review one system you built and redesign it with what you know now.
Related interview prep
Cloud building blocks.
Open →QuestionsSQL interview questionsJoins to window functions, with real output.
Open →RoleSoftware engineer interview questionsThe questions engineers prepare for.
Open →QuestionsMachine learning interview questionsModels, metrics and attention.
Open →Disclosure: TryHired makes HIRED, a paid job-search program, so we have a conflict of interest when we mention it. This question bank is free, with no TryHired signup.
Questions people ask
What is asked in a system design interview?
How do I prepare for a system design interview?
Do junior engineers get system design interviews?
Is there one right answer in system design?
What is machine learning system design?
How this page is built
TryHired wrote these questions and answers. Every Python block on this page was executed with Python 3.12.10 when the page was built, and the output shown is the real output; the build stops if any example fails. The question list is general practice: it does not predict what a particular employer will ask.
- Interviewers ask different questions for different roles and levels.
- Answers are short explanations, not complete references.
- Real designs depend on requirements and constraints an interview only sketches.
- Estimates use round numbers on purpose; state your assumptions.
- Martin Kleppmann, Designing Data-Intensive Applications (O'Reilly, 2017)
- Amazon Builders' Library, timeouts, retries and backoff with jitter: https://aws.amazon.com/builders-library/timeouts-retries-and-backoff-with-jitter/
- Karger et al., Consistent Hashing and Random Trees (1997): https://dl.acm.org/doi/10.1145/258533.258660
- Google SRE book, handling overload: https://sre.google/sre-book/handling-overload/