Caching, and knowing what to throw away

A cache is a subset of some primary data, kept in a place that is faster and cheaper to reach than the primary. That's it. Not all the data — a subset, chosen by how often it's read and how likely it is to be read next.

The two words doing the work are subset and faster. If you cache everything you've built a second database with worse durability guarantees. If the cache isn't meaningfully faster than the source, you've added a network hop for nothing.

Two situations produce every cache I've ever written:

1. A computation is expensive and the answer doesn't change often.
2. Data is large and the users asking for it are far away.

Google is the first case. A query like "weather today" is searched millions of times a day, and answering it from scratch means crawling, indexing, and ranking. Nobody re-runs that per request. The result goes into a distributed in-memory cache; the next identical query is a cache hit and returns immediately. A cache miss runs the real workflow, then stores the result on the way out.

Netflix is the second. The origin servers holding the actual encoded files sit in a handful of US data centres, but the viewers are everywhere, and a single movie exists at 1080p, 720p, 480p and so on. Serving all of that from origin would be a latency disaster. So popular content is pushed to edge locations — servers placed close to users — and served from there. Which content? Not all of it. That would cost more than the origin storage does. Trend analysis and regional viewing patterns decide the subset.

Twitter's trending topics is the first case again, with a nice property: a trend that took terabytes of tweet analysis to compute stays valid for hours. Recomputing it per pageview would take the servers down in seconds. So it's computed every few minutes on a schedule and parked in Redis, and every user request is a key lookup.

Three levels you'll actually meet

As a backend engineer, caching shows up at three layers. Hardware caching exists too, but you don't drive it.

Network. CDNs and DNS.

Hardware. L1/L2/L3, and — the part that matters here — RAM.

Software. Redis, Memcached, ElastiCache. Called software caching because a library is how you touch it, though it's the hardware layer underneath making it fast.

CDN: the cache with a routing problem

A CDN's job is to serve content from the server closest to the user. The flow:

1. Browser resolves the domain via DNS.
2. The CDN's DNS routes the request to the nearest PoP — point of presence, which is just a cluster of edge servers in one region. Routing considers geography and network conditions.
3. That edge server checks its cache. Hit → serve it. Miss → fetch from origin, store it, serve it.
4. Cached content carries a TTL. When it expires, the next request re-fetches from origin.

A user in New York hits a New York PoP. Deploy a frontend on Vercel and this is what's happening to your static assets — the request never reaches an origin server unless the edge doesn't have the answer.

DNS is caching all the way down

DNS resolution, unwound, is expensive. Your device sends the query to a recursive resolver (your ISP's, or a public one like Cloudflare or Google). The resolver asks a root server, which doesn't know the IP but knows where .com's TLD servers are. The TLD server doesn't know either, but knows the authoritative name server for example.com. That one finally answers.

Three hops for one address lookup. Which is why every layer caches it:

• Your OS keeps a local DNS cache.
• Your browser keeps its own on top of that.
• The recursive resolver caches aggressively — that's most of the win.
• Some authoritative servers cache too.

The lookup you're doing right now almost certainly stopped at layer one or two.

Every "it works on my machine but not on the new server" DNS bug I've hit was some layer of this stack still holding a stale record.

Why RAM

Redis is called an in-memory database because it keeps data in RAM instead of on disk. RAM is called random access memory because reaching any address costs about the same — a directly addressed electrical signal, no mechanical seek. A spinning disk has to physically move a head; even an SSD does more work than that.

The trade you're making is explicit:

	RAM	Disk
Access	Fast, roughly constant	Slower
Capacity	Small	Large
Survives power loss	No	Yes
Cost per GB	High	Low

Volatility and capacity, traded for speed. That's the entire reason a cache is a subset and not a copy — and the reason disk-based storage isn't going anywhere.

Redis handles persistence by writing to disk in the background and reloading into memory on start. But every read and write you issue hits RAM. That's the point.

Two strategies

Lazy caching (cache-aside). Check the cache. Miss → fetch from the database, store it, return it. Nothing is cached until somebody actually asks for it. This is the default, and it's what most application code does.

Write-through. Every write updates the database and the cache in the same execution path. Writes get slower — two systems to update — but the cache is never stale. You never serve old data.

Pick based on your read/write ratio and how much staleness hurts.

Eviction

RAM is limited, so at some point an insert has nowhere to go. The eviction policy decides who leaves.

• noeviction — the default when you configure nothing. The write fails with an out-of-memory error. Rarely what you want.
• LRU, least recently used — evict the key untouched for the longest. Tracks when.
• LFU, least frequently used — evict the key with the fewest accesses. Tracks how many times. A key hit 5 times loses to one hit 23 times, regardless of recency.
• TTL-based — evict whichever key is closest to expiring anyway.

LRU and LFU disagree in a useful way. A key hammered once during a spike then abandoned survives LFU and dies under LRU. Pick the one whose mistake you can live with.

Where this lands in a real backend

Expensive queries. A dashboard query with six joins and an aggregation, hit on every page load. Cache the result with a one-hour TTL. Amazon does the same with product details and prices — a MacBook's description doesn't change during a sale, but a million people load it.

Sessions. After login you store a session token somewhere, and every subsequent request reads it. Putting that in Postgres means a database round trip on every single authenticated call. Redis instead.

External APIs. Your backend calls a weather API. Without a cache, every frontend request becomes an outbound call — you hit the rate limit, or the bill. Weather doesn't change in seconds. Cache it for an hour.

Rate limiting. The middleware pulls the client IP out of X-Forwarded-For, increments a counter keyed on that IP, and returns 429 past the threshold. That counter is written on every request that reaches your server. Put it in Postgres and you've added 20ms to every API call and flooded your database with bookkeeping.

Notice what's common: each one is read-heavy with infrequent writes, and each one turns a computation or a round trip into a key lookup. When you find that shape in your own code, you've found a cache.