QIIUB · API Caching — Design (ADR-0014)

Why we cache

Not sale volume. The authentication hot path against the one database that cannot be sharded.

Sales never touch this cache. They are captured on the POS, offline, against local SQLite, and reach the cloud later through sync. The web apps — qiiub-admin and qiiub-portal — are back-office: purchase orders, product management, receiving, dashboards, administration.

What the cache actually protects is tenant resolution and the user-active check. Every authenticated portal or admin request runs MerchantResolutionMiddleware, which turns the token's mid/pid claim into an internal id and confirms the user is still active. Both are reads against Central DB.

The cache exists so that repeated answers to the same question cost microseconds of memory instead of a round-trip to a database every tenant shares.

RequestJWT · mid / pid

›

Resolve tenantPublicId → Id · 5 min

›

User-active check60s

›

Central DBsingle · unshardable · on miss only

Central is the chokepoint. Merchant data scales by sharding across 12–16 databases, so load spreads. Central — the single identity and tenant store — cannot be split that way, so its load is the sum of every tenant's auth traffic, and a slow Central degrades every portal at once. Caching its hot reads is the primary lever.

Terminals are not on this path. DeviceTokenAuthHandler (ADR-0115) sets the tenant server-side and bypasses the middleware. Refresh tokens are DB-backed but touched only on refresh, minutes apart.

How a read works

Every cache site uses in-box HybridCache (Microsoft.Extensions.Caching.Hybrid), configured with L1 only — an in-process memory cache, no distributed tier. One registration in Infrastructure/DependencyInjection.cs, and no AddMemoryCache(): AddHybridCache() registers the IMemoryCache it uses for its own L1.

Reads go through one call

A site asks for a value and supplies a factory. On a hit the factory never runs; on a miss it runs once, no matter how many requests miss the same key at the same moment.

var resolution = await _cache.GetOrCreateAsync(
    QiiubCache.MerchantKey(publicId),          // key format lives in one place
    (Resolver: this, PublicId: publicId),
    async (state, factoryCt) => { … },          // runs only on a miss
    QiiubCache.ResolutionPositive,              // TTL lives in one place
    tags: null,
    ct);

A read resolves in one call. Deduplicating concurrent misses — stampede protection — is the capability the hand-rolled caches lacked, and the main reason for the migration.

What happens when the cache is empty

It starts empty on every boot and is lost on restart or redeploy, and that is safe by design. A missing entry only means the next read goes to the database and stores the answer again. The cache is never the source of truth — it only saves asking the same question twice. Losing it costs a brief burst of database reads, never correctness.

Conventions live in one class

Infrastructure/Caching/QiiubCache.cs is the single source of truth for every TTL, key format and tag. No site inlines a duration or a key string, so the whole convention is reviewable on one screen and a typo cannot drift between sites.

Keeping it fresh

17 endpoints drop cache entries when they change something cached. Two rules govern all of them.

Invalidate after the change is committed

Never before. Between an uncommitted write and its commit, a concurrent request can re-read the old row and re-cache the pre-change value for another full TTL. For most endpoints that means immediately after SaveChangesAsync; the two platform-user endpoints wrap their change in a transaction, so there it means after CommitAsync.

By tag where the key set is unknown, by key everywhere else

Deactivating a user globally has to drop that user's entries across every merchant they belong to, plus their partner and sysadmin entries — without enumerating merchant ids. Every user-active entry therefore carries a user:{userId} tag, and one RemoveByTagAsync clears them all.

Email templates are the second tag site, for the same reason. A language with no row of its own renders the English fallback — and that English content is cached under that language's key. So editing the English row changes what every untranslated language renders, and the renderer cannot enumerate which languages happen to be cached. Every template entry therefore carries a template:{key} tag, including the negative one: an untagged "no such template" would survive the tag eviction and keep asserting the template does not exist for its whole 60s.

The remaining sites always know their exact key, so they use RemoveAsync; a tag there would be extra machinery for nothing.

The tag exists for exactly one case: a change whose full reach the endpoint cannot list. Sites that always know their key use RemoveAsync instead.

Precondition: one API instance

With L1 only, invalidation is process-local — it clears the cache of the instance that served the mutating request. That is why the immediacy described here holds today: production runs a single instance. Scaling out re-opens a staleness window on every other instance, bounded by each site's TTL (60s for user-active, 5 minutes for tenant resolution). Going multi-instance is therefore the trigger to revisit the distributed tier, not an independent decision.

Every eviction in this design clears one process. That is why "more than one instance in production" is the trigger to re-open the distributed tier — the guarantee changes shape, not just the performance.

What deliberately does not invalidate

Create endpoints for merchants, partners and users. A negative entry can only exist if the identifier was queried before the row existed, and these are addressed by server-generated TypeIDs, so no caller can hold one beforehand.
CreateTemplate is the exception and does invalidate — a template key is a fixed system value like magic-link, not a generated id, so a failed render really can leave a negative entry under the key that is about to exist.
Update endpoints other than merchant and prompt. Per ADR-0085 they do not change state flags; state moves through the activate/deactivate endpoints, which do invalidate.

What gets cached

Site	Caches	Key	TTL	Invalidated by
CentralDbMerchantResolver Tenancy/	PublicId → (Id, CountryCode)	merchant:pid:{publicId}	5 min · 60s negative	Merchant update, activate, deactivate (platform + partner)
CentralDbPartnerResolver Tenancy/	PublicId → PartnerId	partner:pid:{publicId}	5 min · 60s negative	Partner activate, deactivate
CachedUserActiveChecker Tenancy/	user / partner-user / sysadmin active flag	user-active:{merchantId}:{userId} user-active:partner:{partnerId}:{userId} user-active:sysadmin:{userId}	60s	Tag `user:{userId}` — 5 user endpoints
TemplateRenderer Services/Email/	email template by key + language	template:{key}:Email:{lang}:Platform:	30 min · 60s negative	Tag `template:{key}` — template create, update, revert, delete
AiCompletionService Services/Ai/	AI prompt configuration by key	ai:prompt:{key}	5 min · 60s negative	Prompt update

The first three sit on the per-request auth path. The last two are invoked far less often — when an email is rendered or an AI feature runs — and are cached for consistency of pattern rather than because they are hot.

Design decisions

Seven choices define how this cache behaves. Each lists what was chosen, what was rejected, and why — so a future change can weigh the same trade-off instead of rediscovering it.

In-box HybridCache, L1 only

Chosen: Microsoft.Extensions.Caching.Hybrid with no distributed tier. First-party, MIT, on the .NET 10 runtime already in use.
Rejected: Redis as an L2 — the API runs a single instance, so there is no cross-instance state to share; it would add a network service and operational surface for no current gain. FusionCache — more capable (fail-safe, backplane, adaptive TTLs) but a third-party dependency with a single primary maintainer, and its extra features address problems that only appear at multi-instance. Keeping IMemoryCache — no stampede protection and no tag invalidation, which are precisely the two gaps worth closing.
Why: The least to secure and the easiest to debug: one process, one layer, no network service, no third-party code. It also leaves the door open — adding a shared cache later turns this into a two-level cache without touching a single line at any call site.

Tenant-first cache keys

Chosen: user-active:{merchantId}:{userId} — a type namespace, then the tenant identifier, then the user.
Rejected: A bare {merchantId}:{userId} without a namespace: shorter, but a key from another site with the same pair of numbers would collide silently.
Why: Critical Rule #1 puts MerchantId first in every query; keys read the same way, so the discipline is visible in both places. The SystemAdmin key deliberately carries no tenant — a SystemAdmin is global, so the cached value is not tenant-scoped data. The EF predicates were reordered to match.

TTL per data type

Chosen: user-active 60s · tenant resolution 5 min · email templates 30 min · AI prompts 5 min · every negative result 60s. A 5-minute global default acts as a floor for any future site.
Rejected: One TTL for everything. A single short value multiplies Central reads for data that rarely changes; a single long value stretches the revocation window on the one value where staleness is a security concern.
Why: The user-active flag decides whether a revoked user still gets in, so it gets the shortest window. Email templates change by hand, weeks apart. The floor exists so a new cache site that forgets to choose still gets something sane.

Two-phase write for negative results

Chosen: GetOrCreateAsync at the positive TTL, then a second SetAsync at 60s when the factory ran and found nothing — gated on a loaded flag set inside the factory.
Rejected: A single 5-minute TTL for both outcomes: simpler, and safe for legitimate state changes now that invalidation exists — but PublicIds are non-guessable TypeIDs, so every enumeration attempt necessarily produces a negative, and a five-fold longer negative means five times as long holding junk in L1. A single 60s TTL: multiplies tenant-resolution reads against Central by five, which is what the cache exists to avoid.
Why: GetOrCreateAsync fixes the entry's TTL before the factory runs, so a fresh not-found lands at the positive TTL and has to be corrected. The gate is load-bearing: rewriting whenever the value merely is not-found — including on a cache hit — resets the clock on every read, so a key polled more often than its TTL would never expire at all. An ungated draft was checked against a real cache with a shortened TTL and the entry never expired while it was being polled; with the gate it expired on schedule. (Exact call counts depend on the machine and the timing; the difference that matters is expires-on-schedule versus never-expires, and a regression test pins it.)

Cached types are declared immutable

Chosen: [ImmutableObject(true)] on MerchantResolution and on a new CachedTemplate projection. bool and int need nothing.
Rejected: Relying on sealed record alone — measured against the package: a plain record is still serialized. Only the attribute is recognised. Putting the attribute on the EF entity — it would be a lie on a type with public setters, and HybridCache hands one shared instance to every caller. Projecting AiPrompt too — 13 of its fields are read and it is passed whole into the request logger, so the projection would be a near-clone plus a large signature change, on a path that is not per-request.
Why: HybridCache serializes a cached value with System.Text.Json unless it can prove the type is immutable. MerchantResolution is read on every authenticated request, so without the attribute the migration would have added a JSON round-trip to the whole API's hot path — a performance regression dressed as a refactor. CachedTemplate also keeps a BodyHtml of up to 500 000 characters out of the serializer.

The factory owns its DI scope

Chosen: Every cache factory resolves its own scope via IServiceScopeFactory and its own CentralDbContext. AiCompletionService keeps a request-scoped context as well, for its audit-log writes.
Rejected: Using the request's DbContext — capturing a scoped service in work that can outlive its scope. IDbContextFactory, which is the textbook answer for a context whose lifetime differs from the request: rejected because CentralDbContext is registered with interceptors, and a parallel factory registration would have to replicate that configuration — any divergence would produce silently inconsistent auditing. A rare, loud failure is a better trade than a permanent, quiet one.
Why: Stampede deduplication means one factory serves every concurrent miss and is owned by the first caller. If that request aborts, its DI scope — and its DbContext — is disposed underneath the still-running factory, and every waiting request gets an ObjectDisposedException. The dedup that makes the cache faster is what creates the coupling.

Evictions are not cancellable

Chosen: CancellationToken.None for every post-commit eviction and for the negative-TTL correction. Reads still take the caller's token.
Rejected: Passing the request's ct everywhere, which is the repo's general rule.
Why: The rule exists so abandoned requests do not burn work. An eviction is different: it cleans up after a change that has already been saved. Cancelling it does not undo that change — it only skips the cache drop, leaving the row deactivated while the cache still asserts the old value for the rest of the TTL, with nothing logged. That is the exact failure this work closes, reached through a disconnect instead of a bug.

Security rules

A cache key is tenant-scoped whenever its value is. MerchantId or PartnerId leads the key, the same discipline Critical Rule #1 applies to queries. This is the real defence against cross-tenant cache leakage and it does not depend on the library. Platform-level reference data — email templates, AI prompts — is exempt by nature, not by oversight.
Caching an authorization decision creates a staleness window. The user-active TTL stays at 60 seconds for that reason, and every path that revokes access invalidates rather than waiting the window out.
Cached values are treated as read-only. With an immutable type the cache hands the same instance to every caller, so mutating one would corrupt the others. Types that are not genuinely immutable are not declared immutable.
No secrets, no connection strings, no network surface. An L1-only cache adds no infrastructure and nothing to secure. Any future distributed tier gets its own security review: private endpoint, Entra ID auth, TLS, secret in Key Vault.

When to revisit

The durable part of this design is the set of triggers, not the products named in it.

Trigger	What it re-opens
More than one API instance in production	Cross-instance invalidation. Every eviction here is process-local, so a second instance means every other one serves stale data until its TTL expires. Watch the App Service tier: Basic is effectively single-instance, Premium autoscales.
A hosted ecommerce storefront	The first genuinely cloud-native transactional path — per-page-view catalog and price reads, carts, stock counters, checkout idempotency, anonymous traffic. That is the workload where a shared cache and edge rate-limiting are actually justified.
Central read pressure at high merchant counts	Read replicas for resolution reads, in addition to caching. Central scales by replication, not by sharding.
Instant revocation becomes a requirement	A backplane. Neither L1 nor a shared L2 makes invalidation instant across instances; only a push channel does.

Re-validate, don't inherit

When a trigger fires, open a fresh decision and re-check the technology, versions and prices. The comparison that informed this design is a dated snapshot kept in docs/research/; by the time a trigger fires, a better option than the ones considered may exist.

One thing not to do

Never hand-roll a Redis Pub/Sub backplane. Pub/Sub is at-most-once with no persistence and no replay: a network blip, a failover or a rolling deploy means an instance misses an invalidation and serves stale data until its TTL — with no error, no log and no metric. If a backplane is ever needed, use a library that ships auto-recovery.

Glossary

Cache stampede

Many requests missing the same key at the same instant and all rushing to the database for the same value. Protection means only the first one asks; the rest wait and reuse the answer.

L1 / L2

L1 is a cache in one server's own memory — fastest, private to that process. L2 is a shared cache every server reads. QIIUB runs L1 only.

TTL

Time to live: how long a cached copy is trusted before it is fetched again.

Negative result

Caching the fact that something was not found, so a repeated lookup for a non-existent id does not hit the database every time.

Invalidation

Actively dropping a cached copy the moment the real data changes, instead of waiting out the TTL.

Tag

A label on a cache entry that lets a whole group be dropped at once — used here to clear all of one user's entries without knowing which merchants they belong to.

Cache coherence

Whether all running instances agree on a cached value. A problem that only exists with more than one instance.

Backplane

A message channel that tells every instance "this changed, drop it", so invalidation reaches servers that did not handle the request. Neither an L1 nor a shared L2 provides this on its own.

Tenant resolution

Turning the merchant's public id from the token into the internal database record a query needs.

Sharding

Splitting data across several databases so load spreads. QIIUB shards merchant data across 12–16 databases; Central cannot be split that way, which is why it needs the cache.

HybridCache

The caching library QIIUB uses — first-party Microsoft, MIT-licensed, part of the .NET libraries. It provides stampede protection, tag invalidation, and the option of adding a shared second level later without touching any call site. Here it runs with the first level only.

Redis

A separate in-memory data store that runs as its own network service, commonly used as the shared cache several app instances read. QIIUB does not use it: with one API instance there is no shared state to keep, and it would add infrastructure to secure and operate. It is the natural candidate for the second level if the API ever scales out.

FusionCache

A third-party caching library, more capable than HybridCache — it can serve slightly stale data when the database is unavailable, and it ships the backplane HybridCache lacks. It can also be plugged in underneath the same interface, so adopting it later would not change any call site. Not used today: its extra features address multi-instance problems QIIUB does not have yet, and it adds a dependency maintained largely by one person.