AI Agent Prompt Compilation - Technical Documentation

Overview

This document explains how the AI agent prompt compilation works, why web search was incorrectly enabled, and the solution implemented to improve performance.

Architecture

Frontend                    Backend                         AI Chat Service
   │                           │                                  │
   │  POST /compile            │                                  │
   ├──────────────────────────►│                                  │
   │                           │  POST /api/chat/message          │
   │                           ├─────────────────────────────────►│
   │                           │                                  │  LLM Provider
   │                           │                                  ├─────────────►
   │                           │                                  │  (Grok/Gemini)
   │                           │◄─────────────────────────────────┤
   │◄──────────────────────────┤                                  │
   │   Structured conditions   │                                  │

The Problem

The condition compiler was experiencing slow response times (5-10+ seconds) due to unnecessary web and X/Twitter searches being performed on every compilation request.

Root Cause

The AI chat service (/api/chat/message) defaults to enabling web search:

# services/ai-chat-assistant/app/routes/chat.py
class ChatRequest(BaseModel):
    use_web_search: bool = True   # Default: enabled
    use_x_search: bool = True     # Default: enabled

These defaults exist because the chat service was designed for the conversational AI assistant where real-time data is essential (current scores, news, odds).

However, the condition compiler reused this endpoint without overriding the defaults:

// backend/src/services/monitoring/conditionCompiler.ts
const response = await axios.post(`${AI_SERVICE_URL}/api/chat/message`, {
  message: userMessage,
  system_prompt: systemPrompt,
  session_id: `condition-compile-${Date.now()}`,
  // use_web_search and use_x_search NOT specified → default to true
});

Why Web Search is NOT Needed for Compilation

What Compilation Does

The condition compiler transforms natural language prompts into structured JSON:

Input: "Alert me when Virat Kohli scores 50"

Output:

{
  "metric": "batsman.runs",
  "operator": "gte",
  "threshold": 50,
  "filters": {
    "player_name": "Virat Kohli"
  }
}

Everything Needed is Already Available

Input	Source	Needs Web Search?
User prompt	Provided directly	No
Available metrics	System prompt (capabilities manifest)	No
Available operators	System prompt	No
Output JSON schema	System prompt	No
Player/team names	LLM training data	No

Player and Team Name Recognition

Q: How does the LLM know who "Virat Kohli" is without web search?

A: LLMs (Grok, GPT-4, Gemini, Claude) are trained on massive datasets that include cricket players, teams, and tournaments. The LLM already knows:

Virat Kohli is a batsman
Mumbai Indians is an IPL team
The Ashes is a cricket tournament

The LLM's job at compile time is extraction, not verification. It extracts the player name as a filter value.

Runtime Resolution

Actual player matching happens at runtime, not compile time:

// From capabilities manifest
filters: {
  player_name: 'Filter by player name (fuzzy matched)',
  team_name:   'Filter by team name',
}

When the monitor runs:

Resolver gets live match data with actual player names
Fuzzy matches "Virat Kohli" against players in the match
Evaluates the condition if there's a match

Separation of Concerns

Stage	Responsibility	Needs Web?
Compile time	Extract names/values as structured data	No
Runtime	Match against live match data	No (uses live feed)

The Solution

Explicitly disable web search in the condition compiler:

const response = await axios.post(`${AI_SERVICE_URL}/api/chat/message`, {
  message: userMessage,
  system_prompt: systemPrompt,
  session_id: `condition-compile-${Date.now()}`,
  use_web_search: false, // Disable - not needed for parsing
  use_x_search: false, // Disable - not needed for parsing
});

Performance Impact

Metric	Before	After
Compilation time	5-10+ seconds	2-4 seconds
Web search calls	1-2 per request	0
Parsing accuracy	Same	Same (or slightly better)

When Web Search IS Needed

Web search should remain enabled for:

Chat assistant queries ("What's the current score?")
Real-time news requests ("Any injury updates?")
Current fixture lookups ("Who's playing today?")

Files Modified

backend/src/services/monitoring/conditionCompiler.ts - Added explicit use_web_search: false and use_x_search: false

services/ai-chat-assistant/app/routes/chat.py - Chat endpoint with defaults
services/ai-chat-assistant/app/services/llm_service.py - LLM service with search support
backend/src/services/monitoring/capabilities/cricketCapabilities.ts - Cricket metrics manifest
backend/src/services/monitoring/capabilities/punterCapabilities.ts - Punter metrics manifest

Rich Notification Alerts

Problem

Alert notifications were showing poor quality data:

Before:

Batsman 50 or Wicket in 35196653
Unknown has reached 50 runs or a wicket has fallen at 0/0 after 18 overs.

Issues:

35196653 = raw fixture ID instead of "India vs Australia"
Unknown = player name placeholder not resolved
0/0 = score not extracted from event/cache

Root Cause

The buildTemplateVars method in cricketMonitoringService.ts was:

Only looking at immediate event data
Defaulting to "Unknown" for missing values
Using fixture ID as fallback for fixture name
Not leveraging cached match state

Solution

1. Multi-Source Data Resolution

The improved buildTemplateVars now checks multiple sources in order:

// For batsman name:
const batsmanName =
  event.batsmanName || // Direct event field
  event.batsman?.name || // Nested object
  event.striker?.name || // Alternative field name
  matchState?.currentBatsman || // Cached match state
  undefined; // Let interpolate handle gracefully

2. Match State Cache Lookup

Added getMatchState() method that checks multiple Redis cache patterns:

cricket:state:{fixtureId}
cricket:context:{fixtureId}
cricket:match:{fixtureId}

This provides fallback data when event data is incomplete.

3. Graceful Placeholder Handling

Updated interpolate() function:

Removes placeholders with missing/unknown values (instead of showing "Unknown")
Cleans up extra whitespace from removed placeholders
Results in cleaner, more readable notifications

4. Enhanced Alert Content

The fireAlert() method now:

Appends fixture name to title if available
Adds score context to message if not already present
Includes resolved values in triggerData for debugging

Expected Result

After:

Batsman Milestone - India vs Australia
Virat Kohli has reached 50 runs! (156/3 after 18.2 overs)

Data Flow

Event Received
      │
      ▼
┌─────────────────────────────────────────────┐
│           buildTemplateVars()               │
│                                             │
│  1. Extract from event data                 │
│  2. Fallback to cached match state          │
│  3. Format scores/overs nicely              │
│  4. Handle missing values gracefully        │
└─────────────────────────────────────────────┘
      │
      ▼
┌─────────────────────────────────────────────┐
│           interpolate()                     │
│                                             │
│  - Replace {placeholders} with values       │
│  - Remove empty/unknown placeholders        │
│  - Clean up whitespace                      │
└─────────────────────────────────────────────┘
      │
      ▼
┌─────────────────────────────────────────────┐
│           fireAlert()                       │
│                                             │
│  - Enhance title with fixture name          │
│  - Add score context to message             │
│  - Store resolved vars in triggerData       │
└─────────────────────────────────────────────┘
      │
      ▼
    Alert Created

Files Modified

backend/src/services/monitoring/cricketMonitoringService.ts
- Enhanced buildTemplateVars() with multi-source resolution
- Added getMatchState() for cache lookups
- Improved interpolate() with graceful handling
- Enhanced fireAlert() with richer content
- Added helper functions formatOvers() and formatScore()

Backward Compatibility

These changes are fully backward compatible:

All changes are additive (new fallback sources)
If new data sources are unavailable, falls back to previous behavior
No API or data structure changes
Existing monitors continue to work

Chase-Specific Template Variables (2nd Innings)

Problem

Alerts for 2nd innings chase scenarios were showing incorrect data:

Before:

Team close to win: 0 runs needed

Issues:

0 runs needed = chase-specific variables weren't being populated
Missing target score, chasing team, balls remaining

Root Cause

The buildTemplateVars() method wasn't extracting chase-specific fields from event data or cached match state:

target (1st innings total to chase)
runsNeeded / runsRemaining (runs still required)
ballsRemaining (balls left in the chase)
chasingTeam (batting team in 2nd innings)
bowlingTeam (fielding team in 2nd innings)

Solution

1. Extract Chase Fields from Event Data

// Chase-specific info (2nd innings only)
// Only populate when we have evidence of a chase (target exists)
const target =
  event.target ??
  event.chase?.target ??
  event.targetScore ??
  matchState?.target;
const runsNeeded =
  event.runsNeeded ??
  event.runs_remaining ??
  event.chase?.runs_remaining ??
  matchState?.runsNeeded ??
  (target && runs !== undefined ? Math.max(0, target - runs) : undefined);
const ballsRemaining =
  event.ballsRemaining ??
  event.balls_remaining ??
  event.chase?.balls_remaining ??
  matchState?.ballsRemaining;
// Only use battingTeam/fieldingTeam as fallback when target exists (confirms it's a chase)
// Don't assume based on home/away - toss determines batting order
const chasingTeam =
  event.chasingTeam ??
  (target ? (event.battingTeam ?? event.batting_team) : undefined) ??
  matchState?.chasingTeam;
const bowlingTeam =
  event.bowlingTeam ??
  (target ? (event.fieldingTeam ?? event.fielding_team) : undefined) ??
  matchState?.bowlingTeam;

2. Added Chase Fields to Match State Cache Lookup

The getMatchState() method now extracts:

// Chase-specific fields
target: state.target ?? state.targetScore ?? state.chase?.target,
runsNeeded: state.runsNeeded ?? state.runs_remaining ?? state.chase?.runs_remaining,
ballsRemaining: state.ballsRemaining ?? state.balls_remaining ?? state.chase?.balls_remaining,
chasingTeam: state.chasingTeam ?? state.battingTeam ?? state.batting_team,
bowlingTeam: state.bowlingTeam ?? state.fieldingTeam ?? state.fielding_team,

3. New Template Placeholders Available

Placeholder	Description	Example
`{target}`	1st innings score to chase	185
`{runsNeeded}`	Runs still required	42
`{runsRemaining}`	Alias for runsNeeded	42
`{ballsRemaining}`	Balls left in chase	36
`{chasingTeam}`	Team batting in 2nd innings	"Mumbai Indians"
`{bowlingTeam}`	Team bowling in 2nd innings	"Chennai Super Kings"
`{homeTeam}`	Home team	"Chennai Super Kings"
`{awayTeam}`	Away team	"Mumbai Indians"

Expected Result

After:

Mumbai Indians close to win - Chennai vs Mumbai
Mumbai Indians need 42 runs from 36 balls to win (143/4 after 14.0 overs)

Fallback Logic

If runsNeeded is not directly available in event data, it's calculated:

runsNeeded = target - runs; // if both target and current runs are available

Important: We do NOT assume chasingTeam = awayTeam or bowlingTeam = homeTeam. The toss determines batting order, so we only populate these fields when actual chase data is available. If chase data is missing, the placeholder is removed gracefully by interpolate().

Design Decisions

1. No Home/Away Assumptions

Decision: Never assume batting order based on home/away team.

Reason: In cricket, the toss winner decides whether to bat or bowl first. The home team could bat second (chase) or first. Making assumptions like chasingTeam = awayTeam would produce incorrect data ~50% of the time.

2. Guard Chase Variables with Target Check

Decision: Only use battingTeam as fallback for chasingTeam when target exists.

// Only use battingTeam when target exists (confirms it's 2nd innings)
const chasingTeam =
  event.chasingTeam ??
  (target ? event.battingTeam : undefined) ??
  matchState?.chasingTeam;

Reason: During 1st innings, there is no target. The batting team isn't "chasing" anything. Using battingTeam without this guard would incorrectly label a team as "chasing" when they're batting first.

Behavior:

1st innings: chasingTeam = undefined (placeholder removed from alert)
2nd innings with target: chasingTeam = batting team (correct)

3. Graceful Degradation

Decision: Return undefined for missing values instead of fallback text like "Unknown".

Reason: The interpolate() function removes placeholders with undefined values and cleans up whitespace. This produces cleaner alerts:

Approach	Result
Fallback to "Unknown"	"Unknown needs 42 runs"
Return undefined	"needs 42 runs" → cleaned to "42 runs needed"

4. Multi-Source Resolution Order

Decision: Check sources in order of specificity: explicit field → nested object → alternative naming → cached state.

const target =
  event.target ??
  event.chase?.target ??
  event.targetScore ??
  matchState?.target;

Reason: Different data sources use different field names. We try the most common/explicit names first, then fall back to alternatives and cached data.

Files Modified

backend/src/services/monitoring/cricketMonitoringService.ts
- Added chase-specific fields to buildTemplateVars()
- Added chase fields to getMatchState() cache lookup
- Added homeTeam, awayTeam to return object for templates

Enriched Notification Template Variables

Problem

Notifications only had access to basic player names, score, and rates. Match context like venue, format, innings number, ball commentary, phase analysis, and wicket details were available in the system but not extracted for notification templates.

Additionally, getMatchState() was reading from cricket:context:{id} cache but nothing was writing to that key — so the cache always returned null for ball/wicket events.

Solution

1. Cache Context Events (enables everything else)

cricket:context events fire after every ball/wicket/state change via publishContextUpdate() in ballByBallService.ts. These contain full match context (venue, format, teams, innings, score, etc.).

We now cache these in handleCricketEvent():

if (trigger === "state_change") {
  const cacheId = fixtureId || matchId;
  await cache.set(`cricket:context:${cacheId}`, event, 300); // 5 min TTL
}

This makes getMatchState() functional — it already checks cricket:context:{id} but nothing was writing to it.

Why 300s TTL: Context events fire frequently during live matches (after every ball). 5 minutes is generous — if no events arrive for 5 minutes, the match is likely in a break and stale data should expire.

2. New Template Variables Available

All variables are extracted following the existing pattern: event field → nested object → alternative name → cached state.

Match Details (from `cricket:context` events → cache)

Placeholder	Description	Source	Example
`{venue}`	Match venue	context event	"Wankhede Stadium, Mumbai"
`{format}`	Match format	context event	"T20", "ODI", "Test"
`{currentInnings}`	Innings number	context event	1 or 2
`{battingTeam}`	Current batting team	context event	"India"
`{wicketsRemaining}`	Wickets in hand	context event	6
`{matchState}`	Current state	context event	"live", "rain_delay", "innings_break"

Ball Detail (from `cricket:ball` events)

Placeholder	Description	Source	Example
`{outcome}`	Ball outcome	`event.template.outcome`	"six", "four", "dot", "wicket"
`{outcomeEmoji}`	Outcome emoji	`event.template.outcomeEmoji`	"🔥", "4️⃣"
`{commentary}`	Ball commentary	`event.template.commentary`	"Full toss, dispatched over long-on"
`{overBall}`	Over.ball display	`event.overBall`	"15.3"

Wicket Detail (from `cricket:wicket` events)

Placeholder	Description	Source	Example
`{dismissedBalls}`	Balls faced	`event.balls`	42

(Existing: {dismissedBatsman}, {dismissedScore}, {dismissalType} — now also extract from event.playerName, event.wicket.player, event.wicket.type)

Session/Phase Detail (from `cricket:session-update` events)

Placeholder	Description	Source	Example
`{phaseName}`	Human-readable phase	`event.phaseName`	"Powerplay", "Death Overs"
`{oversRemaining}`	Overs left in phase	`event.oversRemaining`	"4.0"
`{projectedRuns}`	Projected phase total	`event.projectedPhaseRuns`	185
`{venueAverage}`	Historical venue avg	`event.venuePhaseAverage`	170
`{formatAverage}`	Historical format avg	`event.formatPhaseAverage`	168
`{deviationPct}`	% above/below average	`event.deviationPct`	"8.5%"
`{trend}`	Performance trend	`event.trend`	"above", "below", "on_track"

3. Data Flow After Changes

cricket:context event arrives
        │
        ├──► cache.set('cricket:context:{id}', event, 300)  ← NEW
        │
        └──► evaluate monitors → fire alerts (has full context in event)

cricket:ball event arrives
        │
        └──► evaluate monitors → buildTemplateVars()
                                    │
                                    ├──► extract from event.template.* (ball detail)
                                    │
                                    └──► getMatchState() → cache.get('cricket:context:{id}')
                                              │
                                              └──► returns venue, format, teams, etc.  ← NOW WORKS

Backward Compatibility

All changes are additive — new fields in return object
Existing template variables unchanged (same names, same extraction logic)
interpolate() already removes unused {placeholders}, so templates that don't use new variables are unaffected
Cache write is fire-and-forget — if it fails, behavior is same as before (null from getMatchState())
No API, schema, or interface changes

Overview​

Architecture​

The Problem​

Root Cause​

Why Web Search is NOT Needed for Compilation​

What Compilation Does​

Everything Needed is Already Available​

Player and Team Name Recognition​

Runtime Resolution​

Separation of Concerns​

The Solution​

Performance Impact​

When Web Search IS Needed​

Files Modified​

Related Files​

Rich Notification Alerts

Problem​

Root Cause​

Solution​

1. Multi-Source Data Resolution​

2. Match State Cache Lookup​

3. Graceful Placeholder Handling​

4. Enhanced Alert Content​

Expected Result​

Data Flow​

Files Modified​

Backward Compatibility​

Chase-Specific Template Variables (2nd Innings)

Problem​

Root Cause​

Solution​

1. Extract Chase Fields from Event Data​

2. Added Chase Fields to Match State Cache Lookup​

3. New Template Placeholders Available​

Expected Result​

Fallback Logic​

Design Decisions​

1. No Home/Away Assumptions​

2. Guard Chase Variables with Target Check​

3. Graceful Degradation​

4. Multi-Source Resolution Order​

Files Modified​

Enriched Notification Template Variables

Problem​

Solution​

1. Cache Context Events (enables everything else)​

2. New Template Variables Available​

Match Details (from cricket:context events → cache)​

Ball Detail (from cricket:ball events)​

Wicket Detail (from cricket:wicket events)​

Session/Phase Detail (from cricket:session-update events)​

3. Data Flow After Changes​

Backward Compatibility​

Overview

Architecture

The Problem

Root Cause

Why Web Search is NOT Needed for Compilation

What Compilation Does

Everything Needed is Already Available

Player and Team Name Recognition

Runtime Resolution

Separation of Concerns

The Solution

Performance Impact

When Web Search IS Needed

Files Modified

Related Files

Problem

Root Cause

Solution

1. Multi-Source Data Resolution

2. Match State Cache Lookup

3. Graceful Placeholder Handling

4. Enhanced Alert Content

Expected Result

Data Flow

Files Modified

Backward Compatibility

Problem

Root Cause

Solution

1. Extract Chase Fields from Event Data

2. Added Chase Fields to Match State Cache Lookup

3. New Template Placeholders Available

Expected Result

Fallback Logic

Design Decisions

1. No Home/Away Assumptions

2. Guard Chase Variables with Target Check

3. Graceful Degradation

4. Multi-Source Resolution Order

Files Modified

Problem

Solution

1. Cache Context Events (enables everything else)

2. New Template Variables Available

Match Details (from `cricket:context` events → cache)

Ball Detail (from `cricket:ball` events)

Wicket Detail (from `cricket:wicket` events)

Session/Phase Detail (from `cricket:session-update` events)

3. Data Flow After Changes

Backward Compatibility