A field guide to the concepts behind WhatsApp, Instagram, YouTube, and every other system built to survive millions of users - diagrams, trade-offs, cheat sheets, and the questions interviewers actually ask.
System design is the process of designing scalable, reliable, efficient, and maintainable software systems that can handle millions of users.
Scalability means handling increased traffic without a drop in performance. There are two directions to scale in.
4 GB RAM ──────▶ 32 GB RAM (one server, more power)
User Requests
|
V
Load Balancer
/ | \
S1 S2 S3A load balancer distributes incoming requests across multiple servers so no single machine gets overwhelmed.
1000 Requests
|
V
Load Balancer
/ | \
S1 S2 S3A cache stores frequently accessed data so repeat requests never have to hit the database.
Client
|
Server
|
Database Client
|
Server
|
Redis
|
Database| SQL | NoSQL | |
|---|---|---|
| Examples | MySQL, PostgreSQL, Oracle | MongoDB, Cassandra, DynamoDB |
| Features | ACID, relationships, transactions | Flexible schema, horizontal scaling |
| Good for | Banking, ecommerce, ERP | Social media, big data, chat apps |
Replication copies data from one database to another so reads can be served from copies while writes stay centralized.
Master
/ \
Replica1 Replica2Sharding splits data across multiple databases so each one only holds a slice of the total.
Shard 1 Shard 2 Shard 3 Users A–F Users G–N Users O–Z
A distributed system can't guarantee Consistency, Availability, and Partition tolerance all at once — you get to pick two.
| Letter | Meaning |
|---|---|
| C | Consistency — every read gets the latest write |
| A | Availability — every request gets a response |
| P | Partition tolerance — keeps working despite network splits |
| System type | Typical choice |
|---|---|
| Banking | CP — consistency over availability |
| Social media | AP — availability over strict consistency |
| SQL | NoSQL |
|---|---|
| MySQL | MongoDB |
| PostgreSQL | Cassandra |
| SQL Server | DynamoDB |
| — | Redis (key-value) |
A CDN caches content in edge locations physically closer to users, so it doesn't have to travel all the way to the origin server.
User (India)
|
Nearest CDN Edge
|
Origin Server (USA)A reverse proxy sits in front of your servers and mediates every request that reaches them.
Users
|
Reverse Proxy
|
App ServersAn API gateway is the single front door that manages every incoming API request before it reaches your services.
Client
|
API Gateway
|
MicroservicesEverything — every feature and module — lives inside one deployable application.
Application ├── Login ├── Orders ├── Payments └── Inventory
The application is split into independent services, each owning a single responsibility.
Login Service Order Service Payment Service Notification Service
A message queue lets services communicate asynchronously — the sender doesn't wait for the receiver.
Producer
|
Queue
|
ConsumerServices react to events instead of calling each other directly, which keeps them loosely coupled.
Order Created
↓
Inventory Updated
↓
Payment Processed
↓
Email SentRate limiting restricts how many requests a client can make in a given window, protecting the system from abuse and overload.
100 requests / minute / client
| Authentication | Authorization |
|---|---|
| Who are you? | What can you access? |
| Login, OTP, password | Admin, user, moderator |
Client Login
↓
Server
↓
JWT Token
↓
Client stores token
↓
Sent with API requestsOAuth lets users log in with an identity they already have, instead of creating a new password for every app.
User → Provider (e.g. Google)
→ Authorization
→ Access Token
→ ApplicationHTTPS wraps HTTP traffic in SSL/TLS encryption so data can't be read or tampered with in transit.
Consistent hashing distributes data evenly across servers, so adding or removing a server only reshuffles a small slice of the data.
A bloom filter is a fast, space-efficient probabilistic structure that tells you an item is definitely absent, or possibly present.
A distributed lock ensures only one server processes a critical task at a time, even across a fleet of machines.
A circuit breaker stops calls to a failing dependency so one broken service doesn't cascade into a full outage.
Closed ──failure──▶ Open ──timeout──▶ Half-Open
▲ |
└──────────────success───────────────────┘| Category | Examples |
|---|---|
| Web Server | Nginx, Apache |
| Load Balancer | HAProxy, AWS ELB |
| Cache | Redis, Memcached |
| Message Queue | Kafka, RabbitMQ, SQS |
| SQL Database | MySQL, PostgreSQL |
| NoSQL Database | MongoDB, Cassandra |
| Monitoring | Prometheus, Grafana |
| Logging | ELK Stack, Loki |
| Containerization | Docker |
| Orchestration | Kubernetes |
| Strategy | What it means | Pros | Cons |
|---|---|---|---|
| Vertical | Add power to one box | Simple, no code change | Hardware ceiling, costly |
| Horizontal | Add more boxes | Grows without limit, fault tolerant | Needs a load balancer, more moving parts |
| Choice | When to reach for it | Example domain |
|---|---|---|
| CP | Pick when correctness matters more than uptime | Banking, payments |
| AP | Pick when uptime matters more than perfect freshness | Social feeds, shopping carts |
| Type | Strength | Reach for it when… |
|---|---|---|
| SQL | Structured, relational, transactional | Banking, ecommerce, ERP |
| NoSQL | Flexible schema, scales horizontally | Social media, big data, chat |
Most large-scale designs are a remix of the pieces above. Here's roughly how they stack.
Client
|
CDN (static assets)
|
Load Balancer
|
API Gateway ── auth, rate limiting
|
Microservices ── message queue between them
/ \
Cache Database (replicated + sharded)Always clarify requirements before drawing a single box — an elegant answer to the wrong problem is still wrong.
State your assumptions out loud. Interviewers care more about your reasoning than a 'correct' final diagram.
Estimate scale early: back-of-envelope numbers for users, QPS, and storage steer every later decision.
There is no perfect design, only trade-offs. Naming the trade-off you're making is more valuable than avoiding one.
Start simple, then scale. A single server with a database is a legitimate first draft — add complexity only as bottlenecks appear.
Bottlenecks usually hide in three places: the database, a single point of failure, and unbounded growth of one resource.
Caching solves read-heavy problems; sharding and queues solve write-heavy problems. Reach for the tool that matches the pattern.
Security, monitoring, and failure handling are not optional extras — bring them up even if the interviewer doesn't ask.