Betfair settlement overlap and missing-result alerts
The bulk settlement job revisits the previous 30 minutes (previously 5 minutes) on each discovery cycle. It still pages all four cleared-order statuses, publishes the cursor and inbox atomically, and uses the existing financial drain. This is a bounded-lateness mitigation, not proof of complete settlement. Results published after the overlap can still be missed. The September incident does not establish whether 30 or 60 minutes would have caught the original publication.
Alert-only monitor
A separate timer examines at most 25 unmatched-result candidates per completed minute, using a persisted (created_at, id) scan position and a database lease shared across instances. Eligible orders have a Betfair identity, are non-mock, remain in a settleable matched status, and were placed at least 30 minutes ago. Exhaustion wraps the scan; errors advance past the batch so a bad response cannot starve later orders. Unresolved orders remain eligible on later passes. No settled-order fetch, recovery enqueue, or wallet mutation is added.
The monitor excludes bets whose inbox already has terminal evidence or outstanding processing. For the rest it confirms market status using at most two sequential status-only listMarketBook calls. Betfair omits closed markets in a mixed open/closed request, so the second call requests only IDs omitted by the first. Absence never means closed. See Betfair listMarketBook.
A market must have been explicitly observed CLOSED at least 30 minutes earlier before paging. The first observation is persisted once in orders.betfair_closed_observed_at, without expiry. It survives a restart or a scan longer than seven days; each order observes its own grace period. Before sending, the monitor rechecks local order state, inbox evidence, and its lease. This is observation-time evidence, not an atomic assertion that nothing can change during Slack delivery.
Alerts use SLACK_ORDER_SUBMISSION_WEBHOOK, the existing #strykr-order-alerts route, without fallback to another channel. One summary includes at most 25 local order references and market IDs. A successful Slack HTTP delivery creates 24-hour per-order suppression; failed delivery is retried on a later pass. Wording says no terminal result is recorded in the inbox and requests investigation; it does not blame Betfair or claim raw response proof.
The monitor starts when PAL and the dedicated webhook are configured. BETFAIR_MISSING_RESULT_ALERTS_ENABLED=false disables it completely; other provided values must be true or false. Disabling/restarting does not alter financial state. The parked recovery PR #1662 remains separate and must be reconciled with these alert helpers before any later merge.
Performance and rollout
The monitor has its own scheduling cursor and never awaits inside placement or bulk discovery. It shares the database, Redis, API client, and event loop, so zero impact is not claimed. Its selection index is created concurrently. Review database query plans and deployment migration completion before enabling it at scale.
At 25 candidates per minute, a pass through N continuously unresolved bets takes at least ceil(N / 25) minutes, plus I/O time. This deliberately conservative rollout is not a million-bet alert SLA: one million unresolved candidates would need about 28 days per pass. Higher-scale alerting requires separately budgeted throughput or market-oriented candidate selection. The existing bulk settlement throughput remains a separate concern. The monitor does not accelerate itself when a backlog grows.
A 30-minute overlap also rereads more provider rows than a 5-minute overlap. Compare bulk duration, page count, cursor lag, database contention, and placement latency against baseline before production rollout; capacity testing has not been performed. Roll back the overlap change and/or disable the monitor if those measurements regress. No live deployment is included in this PR.
Historical evidence, inspected 15 September 2026
Passive production inspection found 1,748 retained inbox entries, with first-observation dates from 17 June through 15 September. Among terminal payload rows dated since 1 August, 1,518 SETTLED rows (actual dates 20 August–15 September) and six VOIDED rows were retained. Only the five SETTLED rows from the reported 13 September incident had first-observation delay above five minutes; those same five exceeded 30 and 60 minutes. Their 14.5-hour delay measures rediscovery, not the unknown moment Betfair first exposed them.
Twelve older June/July SETTLED rows also show delays above five minutes but their enqueue reasons describe day-long discovery windows. They cannot be classified as this same moving-window failure from those fields alone. first_observed_at and the current payload are not a complete immutable event history; re-settlement, historical bootstrap, retention, and replay can affect interpretation. Public Slack search also found earlier DEV-labelled vanished-order alerts, including normal cleared-order convergence, which does not establish this same production failure.
Conclusion: no additional matching case was established in the retained recent inbox evidence. This is not proof that this was the first-ever occurrence, and the original raw HTTP responses were not retained.
Review regression evidence
CI runs the exact three alert migrations through the repository-pinned Prisma CLI on an isolated schema in its disposable PostgreSQL service, verifies index validity and the nullable closure column, and checks that a deliberately multi-statement concurrent-index migration fails. Keep the production concurrent-index migration as one SQL statement. Production migration duration and contention remain unmeasured.
A focused CI scratch-copy proof runs the nine-day scan-gap regression green, injects loss of closure observations after seven days and requires an assertion failure, then restores the exact source and requires green again. The full job suite also covers per-order grace when two orders share a market. No production data or provider calls are involved.