Design a newsfeed (Twitter/Instagram-style)

A quick on-ramp: the words you'll need

Before the design, here are the load-bearing terms — each defined once, in plain English, so the rest of the page stands on its own.

• Feed / timeline — a personalized, time-ordered list of posts from the accounts you follow. "Build a feed" = produce that list for one user.
• Fan-out — the act of taking one new post and spreading it to many destinations (the followers' feeds). "On write" = do it when the post is created; "on read" = do it when a feed is requested.
• Push vs pull — synonyms for fan-out-on-write vs fan-out-on-read. Push pushes the post out to followers ahead of time; pull makes the reader pull posts in on demand.
• Cache — a fast, in-memory store (e.g. Redis) holding pre-computed answers so you don't recompute them on every request. Here, each user's pre-built timeline lives in a cache.
• Hot key / celebrity problem — when one item (one account, one post) gets so much traffic that it overwhelms the single machine or queue responsible for it. Celebrities cause hot writes (fan-out storms); viral posts cause hot reads.
• Denormalization — deliberately storing the same data in more than one place (e.g. copying a post into every follower's feed) to make reads faster, accepting extra writes and storage as the cost. Push is denormalization.
• Pagination / cursor — returning a feed in chunks (a "page" of, say, 20 posts) instead of all at once. A cursor is a bookmark ("give me posts older than this timestamp/id") so the next page picks up exactly where the last one stopped, even as new posts arrive at the top.
• Eventual consistency — a guarantee that all copies of the data will agree eventually, but may briefly disagree right after a change. A new follow taking a few seconds to show up everywhere is acceptable; nobody is harmed by a short delay.

A concrete data model

Designs feel abstract until you write down the actual records. Here are the four pieces of state the system keeps. (A KV store is a key-value database — you hand it a key, it hands back the value; think a giant hash map that survives restarts. A Redis sorted set keeps members ordered by a numeric score, here the timestamp, so reading "newest first" is trivial.)

# 1. Posts — source of truth, in a KV store keyed by post_id
#    Sharded (split across machines) by post_id so load spreads evenly.
post:9f2c  ->  { author: "alice", text: "hi", ts: 1733000000,
                media: "s3://.../p9f2c.jpg" }

# 2. Follow graph — who follows whom (its own service, sharded by user)
followers:alice  ->  { bob, carol, dave, ... }   # used by push (fan-out targets)
following:bob    ->  { alice, ed, celeb_x, ... } # used by pull (whose posts to fetch)

# 3. Timeline cache — each user's PRE-BUILT feed, a Redis sorted set
#    score = timestamp, so range reads return newest-first. Capped ~1000.
timeline:bob  ->  ZSET[ (post:9f2c, 1733000000), (post:7a1e, 1732999000), ... ]

# 4. Celebrity list — the few accounts we do NOT push (pull at read time)
celebrities  ->  { celeb_x, celeb_y, ... }   # e.g. anyone > ~100k followers

client → LB → API → post service → write to KV (posts) + enqueue fan-out
                              ↓
                         fan-out worker → for each follower: prepend to their timeline cache
                                          (skip for celebs — they pull-merge at read)
client → API → timeline service → read user's pre-built timeline + merge celeb posts
                                  → rank → return

The write path, traced end to end

Walk through what happens when Alice (a normal account) makes a post:

1. The client sends the post to the API (via the load balancer). The post service writes one record to the posts KV store (post:9f2c) — this is the single source of truth.
2. The post service enqueues a fan-out job and immediately returns success to Alice. She doesn't wait for fan-out; her post is already durably saved.
3. A fan-out worker picks up the job, asks the follow-graph service for followers:alice, and for each follower prepends (post:9f2c, ts) to that follower's timeline:<follower> sorted set in the cache. This is the denormalization step — one post, copied into many feeds.
4. If Alice were on the celebrity list, the worker would skip fan-out entirely — her followers will pull her recent posts at read time instead, avoiding the write storm.

The read path, traced end to end

Now Bob opens the app and requests his feed:

1. The timeline service reads Bob's pre-built feed: a range query on timeline:bob returning the newest ~N post-ids. Because it's a sorted set in a cache, this is one fast lookup — no merging across thousands of authors. This is the payoff of pushing.
2. It checks which celebrities Bob follows (following:bob ∩ celebrities) and pulls their recent posts directly from the posts store. These weren't pushed, so they must be fetched now and merged into the candidate list.
3. The merged candidate set (~500 posts) goes through ranking — reordered by recency, author affinity, engagement, and post type — then the post-ids are hydrated (the full post bodies and media URLs fetched from the KV store and CDN).
4. Results are returned paginated: only the first page (~20 posts) plus a cursor (a bookmark like "older than ts=1733000000"). When Bob scrolls, the next request sends the cursor and gets the next page, so new posts arriving at the top don't shift or duplicate what he's already seen.

Scaling pitfalls to call out

Beyond the headline concerns above, these are the traps interviewers probe for — naming them unprompted signals depth:

• The threshold is a knob, not a constant. "Celebrity" isn't a fixed number — pick a follower threshold and be ready to tune it. Too low and you pull too much (slow reads); too high and a near-celebrity's push still storms.
• New follows and backfill. When Bob follows Alice, his pre-built timeline has none of her past posts. Either backfill the last few on follow, or accept the feed fills in going forward. Don't silently leave it broken.
• Inactive users waste pushes. Pushing to followers who never open the app burns writes and memory for nothing. A refinement: only maintain timelines for recently-active users; pull (rebuild) on demand for the rest.
• Deletes and edits. A pushed post is now copied across many timelines. Deleting/editing means fixing every copy — or filtering deleted post-ids at read time (cheaper, common choice). This is the cost of denormalization.
• Thundering herd on cache miss. If a popular user's timeline is evicted from cache, many simultaneous reads can all try to rebuild it at once and hammer the backend. Guard with a single-flight lock or request coalescing so only one rebuild runs.