Patterns
Built-in patterns#
Three patterns are available out of the box:
- Frustration — the user expresses annoyance, dissatisfaction, or negative sentiment toward the agent
- Confusion — the user indicates they don't understand or are lost
- Task failure reported — the user reports that the agent didn't complete their request correctly
Built-in patterns cannot be edited or deleted.
Custom patterns#
Create your own patterns to monitor behaviors specific to your use case. Custom patterns use AI-powered classification to detect matches based on your description and optional labeled examples.
Creating a pattern#
Click Create pattern to open the creation wizard, which has three steps.
Step 1: Define the pattern
- Pattern name — a human-readable name (up to 50 characters). Example: "User frustration"
- Unique pattern ID — an identifier auto-generated from the name in kebab-case (up to 50 characters). Example: "user-frustration"
- Description — describe what this pattern detects (up to 350 characters). The more specific your description, the better the detection accuracy.
Trigger conditions control which runs are checked for this pattern:
- When to check — check all runs (default), runs that have any of specific tools, or runs that have all of specific tools
- Metadata filters (optional) — only check runs that match specific metadata key-value pairs, up to 10 filters
Step 2: Label examples (optional)
Improve detection accuracy by labeling recent runs as positive or negative examples:
- Browse runs from the past 7 days
- For each run, mark it as matches (positive) or does not match (negative)
- You can label up to 100 examples
- Filter runs by text search, run type, attributes, or metadata
Step 3: Configure alerts
- Enable Slack alerts — toggle whether to send notifications when this pattern is detected
- Select channels — choose which Slack channels receive alerts (requires a connected Slack workspace with subscribed channels)
- Alert frequency — every detection, threshold-based (e.g. 5 detections within 60 minutes), or daily digest
When you create a pattern, it automatically scans recent historical runs for matches.
Patterns list#
The patterns page shows all configured patterns — both built-in and custom — with their detection counts. Use the search bar to filter by name.
Pattern detail#
Click a pattern to see all runs where it was detected. Each run shows:
- The confidence score of the detection
- Evidence explaining why the pattern matched
- A link to the full run detail
For custom patterns, you can edit trigger conditions and alerts, or delete the pattern.