System Design Interview Questions & Answers

Vertical scaling is simpler (no architectural change needed) but has a hard ceiling -- eventually you can't buy a bigger machine, and a single point of failure remains. Horizontal scaling requires the application to be designed for distribution (stateless servers, a load balancer, a data layer that can also scale out), but it scales further and improves fault tolerance, since losing one of many machines doesn't take down the whole system.

Round robin cycles through servers in order, simple but ignorant of each server's current load. Least connections routes to whichever server currently has the fewest active connections, better suited when request processing times vary widely. IP hash routes a given client consistently to the same backend server, useful when session affinity (sticky sessions) is needed without a shared session store.

Consistency means every read receives the most recent write (or an error). Availability means every request receives a (non-error) response, even if it's not the latest data. Partition tolerance means the system keeps operating despite network failures between nodes. Since partitions will happen in any real distributed system, the meaningful trade-off in practice is CP (favor consistency, may reject requests during a partition) vs. AP (favor availability, may serve stale data during a partition) -- the right choice depends on the specific use case (e.g. a banking ledger leans CP, a social media feed often leans AP).

Treating CAP as 'pick any two' as a permanent, static property of a whole system, rather than a trade-off that can differ by operation or even change dynamically during a partition.

A monolith is simpler to develop, test, and deploy initially -- one codebase, one deployment, in-process function calls instead of network calls. Microservices let different teams deploy independently and scale only the services that need it, but introduce real costs: network latency and failure modes between services, distributed data consistency challenges, more complex testing (integration across service boundaries), and significant operational overhead (service discovery, distributed tracing, versioned APIs between services).

Start with a well-modularized monolith and split out microservices only where there's a demonstrated organizational or scaling need -- premature microservices mostly add operational cost without benefit.

Splitting into microservices before the team or the scale actually justifies it, paying the distributed-systems tax without a corresponding benefit.

If a client sends a 'charge $10' request and times out waiting for a response, it can't tell whether the charge actually succeeded before the timeout -- retrying blindly risks charging twice. Designing the operation to be idempotent (e.g. by requiring an idempotency key that the server checks against previously-processed requests before applying the charge again) makes retries safe, since a duplicate request with the same key is recognized and simply returns the original result instead of repeating the side effect.

Use a client-generated idempotency key for any operation with a side effect (payments, order creation) that a client might legitimately need to retry.

Assuming a retried request is always safe by default, when in fact any operation with a side effect (a charge, an email send, an order creation) needs explicit idempotency handling to be retry-safe.

A database query result that's read far more often than it changes is a good caching candidate -- storing it in Redis or an in-memory cache avoids re-running the same expensive query for every request. A CDN caches static assets (images, JS, CSS) geographically close to users. HTTP caching headers (Cache-Control, ETag) let browsers and intermediate proxies avoid re-fetching unchanged responses entirely.

Cache data proportional to how often it's read versus how often it changes -- caching something that changes every request adds complexity with little benefit.

Caching data without a clear invalidation strategy, leading to stale data being served long after the underlying source changed.

A queue (like a task queue) is well suited to distributing work across a pool of workers, where you want each job processed exactly once by whichever worker picks it up. Pub/sub (like a topic-based event bus) is suited to broadcasting an event to multiple independent consumers that each need to react to it -- e.g. an 'order placed' event notifying a shipping service, an email service, and an analytics service simultaneously, each getting their own copy.

A common pattern is one primary (accepting writes) and multiple read replicas (serving read traffic to reduce load on the primary), with application code routing reads and writes accordingly. Synchronous replication guarantees a replica is up to date before confirming a write, at the cost of added write latency (and reduced availability if the replica is unreachable). Asynchronous replication is faster for the writer but means a primary failure right after a write can lose that write if it hadn't yet propagated to any replica.

Be explicit about which consistency/latency trade-off a given piece of data actually needs -- not every write requires synchronous, cross-region replication.

Reading from a replica immediately after writing to the primary and being surprised the read doesn't reflect the write yet, due to normal (and expected) asynchronous replication lag.

It's acceptable when a brief window of staleness doesn't meaningfully harm the user experience or business correctness -- e.g. a 'like count' on a social post being off by a few for a couple of seconds is fine, whereas an account balance being briefly wrong generally isn't. Systems favoring availability and partition tolerance under CAP typically land here, accepting temporary inconsistency in exchange for always being able to serve a response.

Match the consistency model to what the specific data actually requires -- not every piece of data in a system needs the same consistency guarantee.

Applying eventual consistency uniformly across an entire system, including data (like financial balances) where a stale read has real, user-visible consequences.

Interviewers are listening for concrete specifics rather than generic buzzwords: what metric showed the system was struggling (latency, error rate, queue depth), what investigation pinpointed the actual bottleneck, what trade-offs were considered before choosing a fix, and what the measured improvement was afterward. Being able to explain why the alternative approaches weren't chosen shows deeper judgment than just describing the final solution.

Core flow: on creation, generate a short code (a simple approach is a monotonically increasing id encoded in base62 to keep codes short; a hash-based approach needs collision detection/retry), store the mapping (short_code -> long_url) in a datastore optimized for fast key lookups, and on a GET to the short URL, look up the mapping and issue an HTTP redirect. Given the read-heavy nature (far more redirects than creations), a cache in front of the datastore for hot short codes, and a CDN/edge layer for the redirect itself, significantly reduces latency and backend load at scale.

Design for the actual read/write ratio -- URL shorteners are read-heavy, so optimize the redirect path (caching, fast lookups) more than the creation path.

Using a purely random short code without checking for collisions against existing codes, risking two different long URLs silently sharing one short code.

A token bucket algorithm (each client has a bucket that refills at a fixed rate and each request consumes a token) smooths bursts better than a naive fixed window, which can allow 2x the intended rate right at a window boundary. Storing the counter/bucket state in Redis (using atomic INCR or a Lua script for correctness under concurrency) rather than in-memory on each API server is essential once there's more than one server instance, or each instance would enforce its own separate limit.

Enforce the limit atomically in the shared store (e.g. a Redis Lua script) to avoid race conditions where concurrent requests both read the counter before either increments it.

Implementing rate limiting with separate read-then-increment steps that aren't atomic, letting concurrent requests both slip through just under the limit due to a race condition.

An application publishes a generic 'send notification' event (with the channel, recipient, and template data) onto a queue rather than calling the email/SMS/push provider synchronously in the request path -- this keeps the triggering request fast and resilient to a slow downstream provider. Separate worker pools per channel consume from the queue, apply retry with backoff for transient provider failures, and record delivery status, so a spike in email failures doesn't back up SMS or push delivery.

Make notification delivery asynchronous and decoupled from the triggering business action, so a slow or down notification provider never blocks the core user-facing request.

Storing large binary blobs directly in a relational database bloats the database and doesn't scale well for storage or bandwidth. An object store is purpose-built for this, and generating a pre-signed upload URL lets the client upload the file bytes directly to the object store while the application server only handles the lightweight metadata write, avoiding the app server becoming a bandwidth bottleneck for large files. For sync/versioning features, a metadata table tracking file versions and change events (rather than re-uploading the whole file on every edit for large files) supports efficient incremental sync.

Keep large binary data out of the primary transactional database -- use purpose-built object storage and let the application layer manage only metadata and access control.

Proxying every file upload/download through the application server instead of using pre-signed URLs, turning the app server into an avoidable bandwidth bottleneck.

Each user's client holds a WebSocket connection to one of many connection servers behind a load balancer. Since two users chatting might be connected to different connection servers, a shared pub/sub layer lets a message published on one server's channel reach subscribers connected to any other server, so message delivery doesn't depend on both users sharing the same connection server. Message history is persisted separately (a database optimized for the read pattern of 'recent messages in a conversation') so it survives connection server restarts and supports scrollback/search independent of the real-time delivery path.

Separate the real-time delivery path (low-latency, ephemeral) from the durable storage path (message history) -- they have very different performance and consistency requirements.