Cache invalidation for product catalogs: where store speed is decided

Speed is the easy half of caching

Caching a product catalog is not difficult. Set a TTL, point the CDN at the origin, watch the hit ratio climb. The reason stores keep switching caching off, or keep a cache with a 60-second TTL that buys almost nothing, is that a cached page can be confidently wrong. It says a jacket costs $180 when the price moved to $210 an hour ago. It says the sofa is in stock after the last unit shipped. Either mistake costs more than the milliseconds the cache saved.

So the interesting work is not making pages cacheable. It is deciding what a cache entry is allowed to be wrong about, and building the invalidation path that makes that promise hold across a CDN with a hundred points of presence.

Freshness is a per-field decision, not a per-page one

A product page mixes data with wildly different tolerances. Treating the whole page as one unit forces you to pick the strictest requirement for everything, which is how stores end up with uncacheable pages and an origin doing all the work.

Price and stock are the fields that get people fired. A price mismatch at checkout is a consumer-protection problem in most jurisdictions, and a stale in-stock badge turns into a cancellation email and a chargeback. Those fields either come from a fragment fetched after the HTML renders, or they carry a cache lifetime measured in single-digit seconds.

Descriptions and specifications tolerate minutes to hours. Marketing copy tolerates a deploy. Images should never be cached by TTL at all: give them content-hashed filenames, mark them immutable, and let the browser keep them for a year. Reviews, ratings and inventory counts sit somewhere in between.

There is one consequence worth spelling out. Any page that displays a price has to be invalidated when a price changes, and a product appears on far more pages than its own URL. Category listings, brand pages, sale collections, search results, recommendation rails, the sitemap, the shopping feed. Miss one and the failure is silent, which is the worst kind.

Tag the objects, not the URLs

URL purging works when one change affects one URL. It falls apart the moment a single SKU appears in forty places, because now the invalidation job has to reconstruct that list correctly every time, forever.

Tag-based invalidation flips the direction. Each cached response carries a header listing the entities it depends on, and a change purges by entity rather than by address. On Fastly that is the Surrogate-Key header, and the purge removes every cached object carrying the key regardless of URL.

Surrogate-Key: product-8842 cat-jackets brand-acme price-tier-3

curl -X POST https://api.fastly.com/service/$SVC/purge \
  -H "Fastly-Key: $TOKEN" \
  -H "Surrogate-Key: product-8842"

The support is no longer a Fastly differentiator. Cloudflare opened purge-by-tag to every plan in April 2025 and translates Surrogate-Key into its own Cache-Tag format, so origin code does not have to change. CloudFront added cache-tag invalidation on 29 April 2026 with under five seconds of P95 propagation. Varnish has had the concept as secondary keys for years, and it exists because the alternatives do not scale: a purge removes exactly one object, and a ban matches by pattern but has to be evaluated against every object in the cache, so a large ban list costs more CPU the bigger the cache gets.

Two limits shape the tag design. Objects have a tag budget, 50 per object on CloudFront, so tags have to be chosen for grouping rather than for completeness. And purges are rate limited, 100,000 per hour per customer on Fastly with each key counting as one purge no matter how many objects carry it. That second number is generous only if the taxonomy is coarse enough. Tagging every SKU individually and purging forty thousand keys after a feed import will hit the wall.

A catalog import is a purge storm

The nightly supplier feed touches tens of thousands of SKUs at once. A naive invalidation path fires one purge per changed row, the CDN rate limiter starts returning 429s, the retry loop makes it worse, and by morning the cache is half empty and the origin is absorbing everything.

Three controls fix it.

Coalesce first. Collect dirty entity ids for a couple of seconds, deduplicate them, and send one batched request. A feed import that changes 40,000 rows usually changes far fewer distinct tags, because a tag like cat-jackets or brand-acme covers thousands of rows at once. Batch endpoints exist for exactly this.

Then choose soft or hard purge per field. A hard purge deletes the object, so the next request pays a full origin round trip. A soft purge marks it outdated and lets the edge serve it while it revalidates in the background, which is what stale-while-revalidate from RFC 5861 does for browser-side caches. Prices and stock warrant a hard purge. Copy changes do not.

Finally, control the timing. Invalidating 40,000 objects at 2 a.m. means the cache is cold at 8 a.m. when the traffic arrives, unless something warms it. Schedule the import so the invalidation finishes close to the traffic ramp, or pre-warm the top few thousand URLs by requesting them from the origin before the purge lands.

The origin has to survive the rebuild

A cache that empties all at once does not protect the origin, it points a hose at it. Cache stampedes happen because entries written at the same moment expire at the same moment, and the first requests after expiry all miss on the same key.

Request coalescing, sometimes called single flight, is the cheapest fix: the first request for a missing key takes a lock and generates the response, and concurrent requests for the same key wait for it instead of each hitting the database. TTL jitter spreads expiry so objects do not all die together. Pre-emptive refresh rewrites an entry shortly before it expires rather than after. And stale-if-error keeps serving a known-good response when the origin returns 5xx, which turns the cache into an availability feature rather than a speed feature.

Cache-Control: public, max-age=60, stale-while-revalidate=600,
  stale-if-error=86400

The CDN layer needs its own defence, because every edge POP misses independently. A shield tier, an origin-facing cache that the POPs collapse into, is the standard answer. Without one, a cold key in a hundred locations is a hundred origin requests for the same bytes.

Do not put the cart in the cache

The classic failure is a cache key that includes cookies. It produces a hit ratio near zero, so someone removes the cookie from the key, and then a page rendered with one customer's cart gets served to thousands of others. The second version is worse than slow pages, and it usually surfaces as a support ticket about a stranger's name in the header.

The cache key should be the path plus a small set of normalized query parameters. Anything user-specific belongs in a fragment fetched after the HTML renders, or in a response that is explicitly not cacheable. Adding Vary headers as a fix makes it worse: Vary: Cookie, Accept-Encoding, User-Agent multiplies keys until the cache stores one copy per visitor and holds nothing.

Personalized pricing and member-only discounts make this a business rule, not just a caching rule. If the price depends on who is asking, the page cannot be shared, and the honest design is a static shell with the price injected per request.

Measure the cache, not the page

Hit ratio on its own tells you very little. A store at 96 percent overall can be at 30 percent on HTML and 99 percent on images, and the 30 percent is the number that decides how much database capacity you need. Split the metric by content type before drawing conclusions.

Track purge propagation separately. Pick a canary URL, change something, and record how long until every region reflects it. That number is the real staleness budget, not the TTL you configured. The Cache-Status response header, defined in RFC 9211, reports hit, miss and stale per request at each layer, which is the fastest way to find out which layer is answering.

Also count how often the edge serves stale content and how often stale-if-error fires. A high stale count on product pages means the freshness budget is wrong, or a purge loop is churning. Stale-if-error firing regularly means the origin is unhealthy and the cache is hiding it.

A checklist for the invalidation path

• List the fields a cached page depends on, and the maximum staleness each one tolerates in seconds.
• Move volatile fields (price, stock) into a fragment or give them a lifetime under ten seconds.
• Tag every response with entity keys, and keep the tag count per object inside the CDN's limit.
• Coalesce purge requests, and batch them so a feed import produces hundreds of purges rather than tens of thousands.
• Use hard purges for price and availability, soft purges for copy and imagery.
• Confirm stale-while-revalidate and stale-if-error are set on HTML, not just on assets.
• Add request coalescing in the application layer for anything expensive to rebuild.
• Keep cookies and user identity out of the cache key, and test with a logged-in session before trusting the hit ratio.
• Measure purge propagation with a canary URL in more than one region.

Caching is bookkeeping

Cache invalidation has a reputation as one of the hard problems in computer science. In a product catalog it is mostly bookkeeping: knowing every place a SKU appears, tagging those responses so a single change can find them, and being honest about which fields may lag. The performance work is the part a junior developer can finish in an afternoon. The invalidation path is the part that decides whether the cache stays switched on.

Stores that get this right rarely talk about their CDN. They just stop having incidents where a customer sees yesterday's price, and the origin quietly drops to a fraction of the load it used to carry.