Carousel Feed Architecture — Before and After

How the home-page rails (LIVE MARKETS / UPCOMING MATCHES) are built, what changed in PR #1358, and why each decision was made.

Endpoint: GET /api/fixtures/carousel/grouped Consumers: /home rails (via useCarouselGrouped). The flat GET /api/fixtures/carousel shares the same loader and serves the Sports-page top carousel.

---

1. What this endpoint returns

{ "top":     [ /* 20 cross-sport ranked fixtures */ ],
  "bySport": { "27": [ /* up to 10 */ ], "10": [ ... ] } }

top is split client-side by status === 'live' into the two rails (strykr-fe/src/app/home/page.tsx). bySport backs the sport-avatar filter row. One request replaces an older 1 + N fan-out (one call for the top rail plus one per visible sport).

---

2. The problem, as measured

Three consecutive anonymous calls on dev, before any change:

4.912s   5.325s   5.310s

Zero variance across runs = nothing was cached anywhere. Per request:

Phase	Work	Cost
Fixture discovery	1 in-play + 7 daily listMarketCatalogue per sport × 20 sports ≈ 160 REST calls	see note
Market books	listMarketBook for every discovered market — 839 books to render 34 cards, in batches of 40 with a 100 ms pause between batches	dominant

Note on the phase split. An early version of this analysis attributed ~2.8 s to discovery. That was wrong: it counted 160 REST calls as additive wall-clock, but loadCarouselFixtures fans out with Promise.all over sports, so the sweeps run concurrently. Discovery wall-clock was ~0.3–0.9 s; the book fetch was the real cost. The call count was right, the latency attribution was not.

The frontend polls this every 10 s per open tab, and the book fetch is per-request and unshared — so N users meant N × 839 book requests, Betfair throttled, and everyone waited. Measured: 5 concurrent requests took ~7 s each.

---

3. Before → After

Measured on dev1 (same box, same endpoint):

	Before	After
Sequential median	~2.80 s	0.475 s
Sequential min / max	2.57 / 2.93 s	0.445 / 1.501 s
5 concurrent, each	6.8 – 7.3 s	1.40 – 1.55 s
10 concurrent, each	—	2.23 – 2.68 s
Market books per request	839	72
Upcoming catalogue calls per request	140	0 (Redis)

---

4. The five architectural changes

4.1 A caching layer under discovery, with a designated writer

BetfairDiscoveryCache sits between the Betfair adapter and listMarketCatalogue.

The load-bearing decision is what it excludes:

Discovery call	Per sport	Cached?	Why
fetchUpcomingByDay (7 daily calls)	7	Yes	7/8 of the cost; next week's fixture list is static
fetchLivePrimaryMarkets (inPlayOnly)	1	No	it is the live-status signal; a stale copy would park a just-started match in the Upcoming rail for the whole TTL
Prices (listMarketBook)	—	No	never cached, so nothing here can put a stale price on screen

One designated writer. providerCatalogueRefreshJob already swept every PAL sport on the cadence the cache needed, so it became the writer via discoveryCache: 'refresh' — no new job. That flag is load-bearing: a reader would satisfy itself from the very key it exists to renew, the key would expire once and never be rewritten, and the carousel would silently return to the full sweep with nothing failing. There is a test asserting the argument itself.

TTL must exceed the writer interval. At TTL == interval (both 300 s) keys expired just before renewal and real requests paid the sweep — measured: only 13–18 of 20 keys present, TTLs 68–134 s instead of near-300. Now 600 s against a 300 s job.

Do not raise the TTL to hours or days. While the job is healthy the TTL barely matters; it is precisely what bounds a writer outage. Short TTL degrades to "slow but correct"; a long TTL degrades to "fast but silently stale" — new matches never appear, finished ones drop off on their own, so the rails quietly shrink and never grow, with nothing failing.

4.2 Pricing became a capability, and moved out of discovery

Previously "fetch fixtures" and "fetch their prices" were one inseparable operation. Now pricing is a separate, list-scoped, cross-sport capability:

• BetfairAdapter.enrichFixturesWithMarkets — resolves each fixture's Match Odds market from the in-process catalogue map and issues one batched listMarketBook
• BifrostAdapter.enrichFixturesWithMarkets — reads BifrostCache; no network at all
• ExchangeCoordinator.enrichFixturesWithMarkets — dispatches per provider from the bfs_ id convention

This decouples how many fixtures exist from how many we pay to price. It is capability-gated: a provider without list enrichment gets its fixtures back unenriched with a warning, never silently routed to another provider's enrichment (financial-security §7).

4.3 Live-ness changed its source of truth

Live status used to be derived from a fetched price book (book.inplay === true, RCA 2026-07-04). Once pricing moved after ranking that signal did not exist yet, so every started match would have ranked not_started and been dropped — emptying the live rail.

It now derives from in-play discovery membership: a fixture whose catalogue came from the inPlayOnly scan is live. Equally direct provider evidence, one call per sport, no book needed. Opt-in via deriveLiveFromInPlayScan, promote-only (never demotes, never infers terminal).

4.4 The display overlay now runs at two points

applyTournamentMarketDisplayToFixtures does two jobs: stamping the tournamentTier the ranker buckets on, and gating individual markets. Ranking moved ahead of pricing, so those two jobs now happen at different times:

• pass 1 — before ranking: tier, visibility, names (no markets exist, so its market half idles)
• pass 2 — after pricing: the per-market dictionary bettability gate

Safe to run twice because resolution keys off the stable (provider, providerFixtureId) pair, which the overlay never rewrites — pinned by four tests, the load-bearing one asserting that pass 2 gates markets attached after pass 1.

trustExistingTier. buildTournamentGroupAggregates unions over the tournaments present in the input, so the result is set-dependent. Pass 1 sees 20 sports; pass 2 sees the ranked candidates. A tournament inheriting its tier through a union with a match that did not survive ranking re-forms alone, loses its tier, and is dropped after ranking with no log. Pass 2 therefore reuses pass 1's stamped tier and name. There is a test reproducing the dissolve in both directions.

4.5 Everything is behind an opt-in flag

discoveryCache, deferMarkets, deriveLiveFromInPlayScan, trustExistingTier — all optional, all absent by default.

Surface	Cached discovery	Deferred pricing
/carousel/grouped — Home rails	yes	yes
/carousel — Sports-page top carousel	yes	no
/fixtures/live, /fixtures/upcoming — Live page, Sports fixture lists	no	no
AI chat tools, watchlist service	no	no

Blast radius stays small, and a rollback is a config change rather than a revert.

---

5. Deliberate trade-offs

Bifrost replacement is off for the carousel. mergeSupplementFixtures only considers replacing a Betfair row with a Bifrost one when includeMarkets === true, because the decision needs prices. deferMarkets passes false, so on compose sports (cricket, soccer) a match whose Betfair book is suspended while Bifrost's is open is now dropped rather than rendering with Bifrost prices. Accepted after measuring prod: 124 supplement merges over 6 h, zero replacements — the path is dormant. The skip is counted and logged so it is visible. Do not "fix" it by forcing includeMarkets: true, which reinstates the full per-request book fetch. The correct fix, if it ever matters, is to resolve replacement after enrichment where prices exist.

Rails can come back short. We rank 2× the rail size and the playable-market filter removes candidates whose book came back dead. Drop causes are correlated per sport, so the realistic failure is a whole rail collapsing rather than the odd card. Reported via logger.warn and meta.shortRails.

Partial sweeps are never stored. fetchUpcomingByDay tolerates per-day failures so one bad day cannot blank a sport — which makes a sweep missing 6 of 7 days indistinguishable by shape from a genuinely quiet sport. It now reports failedDays, and an incomplete sweep is returned to its caller but never written, leaving any existing entry intact. An older complete list beats a newer broken one.

---

6. Known limits

• Concurrency still scales linearly. 5 → 10 users roughly doubles latency, because the book fetch is per-request and unshared. A shared response cache would make N users cost one fetch; prod data suggests 113 of 114 users would receive an identical response.
• In-play prices are already streamed and still re-fetched. autoStreamLiveMarkets keeps every in-play book fresh in BetfairCache, and enrichment still calls listMarketBook for them. Reading from memory would make the live rail's prices free.
• Home rails receive no WebSocket odds. useWebSocket is not mounted on /home, and handleOddsUpdate only patches the fixture-detail query cache, never fixtureKeys.carousel(). The rails rely entirely on the 10 s poll.
• The candidate set is uncapped. Worst case is 20 sports × 10 × 2 + 40 ≈ 440 ids → ~11 sequential book batches. Observed 52–76 in practice.