Variables & Functions
Reference of router metrics, variables, filters, operators, and target formats.
Available Metrics
LangDB provides a rich set of real-time metrics for making dynamic, data-driven routing decisions. These metrics are aggregated at the provider level, giving you a live view of model performance.
ttft
Time to First Token (ms)
450
Optimize for user-perceived speed
tps
Tokens Per Second (output_tokens/latency)
300
Measure model generation speed
error_rate
Fraction of failed requests
0.01
Route around unreliable models
latency
Average end-to-end response time (ms)
1100
Route based on overall performance
requests
Total number of requests processed
1500
Monitor traffic and usage patterns
cost
Estimated request cost (USD)
0.02
Track and control spend in real time
Available Variables
Variables provide contextual information from the incoming request and user metadata. Unlike metrics, they are not performance indicators but are essential for conditional logic.
User Information
extra.user.id
User identifier from request
"u-12345"
Custom user identification
extra.user.name
User name from request
"john.doe"
Personalization
extra.user.tier
User tier from request
"premium"
SLA-based routing
extra.user.tiers
Array of user tiers from request
["premium", "beta"]
Multi-tier access control
extra.user.*
Any custom field in extra.user
Various
Custom business logic
Request Metadata
metadata.ip
IP address of the requester
"203.0.113.42"
Geo-fencing, fraud detection
metadata.region
Geographical region (based on IP)
"EU"
Data residency, compliance
metadata.user_agent
Client application
"Google ADK/ CrewAI"
Agentic library detection
metadata.user_id
LangDB user ID
"langdb-u-12345"
Internal user identification
metadata.group_id
LangDB cost group ID
"grp-456"
Cost group identification
metadata.group_name
LangDB cost group name
"development"
Cost group-based routing
metadata.tags.{tag}
LangDB user tags
"premium"
User classification
Pre-Request Interceptor Results
pre_request.{interceptor_name}.passed
Whether interceptor check passed
true
Conditional routing based on checks
pre_request.rate_limiter.passed
Rate limit check result
true
Enforce usage quotas
pre_request.semantic_guardrail.passed
Content policy check result
false
Block policy violations
pre_request.{guardrail}.result.topic
Detected topic from guardrail
"billing"
Topic-based model routing
pre_request.{guardrail}.result.*
Any custom result from interceptor
Various
Custom business logic
Provider Performance Metrics (for Filtering)
provider.error_rate
Provider's current error rate
0.02
Avoid unreliable providers
model_capabilities
Supported features (vision, code)
["vision"]
Feature-based routing
model_tags
Model classification tags
["fast"]
Tag-based filtering
Target Configuration
Simple List
Direct array of model names
["openai/gpt-4o", "anthropic/claude-4-sonnet"]
Fixed model selection
Provider Wildcards
All models from specific provider
["openai/*", "anthropic/*"]
Provider-level routing
Single Target
Direct model specification
"openai/gpt-4o"
Fixed routing to one model
$any Pool
Pool of models with selection logic
{ "$any": [...], "sort_by": "price" }
Dynamic model selection
Model Target Formats
Exact Model Name
Specific model identifier
"openai/gpt-4o"
Direct routing
Provider Wildcard
All models from provider
"openai/*"
Provider-level failover
Sorting Options
"sort_by": "price", "sort_order": "min"
Picks the cheapest model
Cost control, bulk/low-priority tasks
"sort_by": "price", "sort_order": "min"
"sort_by": "price", "sort_order": "max"
Picks the most expensive model
Premium users, highest quality needed
"sort_by": "price", "sort_order": "max"
"sort_by": "ttft", "sort_order": "min"
Picks the fastest model (time-to-first-token)
VIP, real-time, or user-facing tasks
"sort_by": "ttft", "sort_order": "min"
"sort_by": "latency", "sort_order": "min"
Picks the lowest latency model
Performance-critical applications
"sort_by": "latency", "sort_order": "min"
"sort_by": "error_rate", "sort_order": "min"
Picks the most reliable model
Mission-critical or regulated workflows
"sort_by": "error_rate", "sort_order": "min"
"sort_by": "tps", "sort_order": "max"
Picks the fastest token generation model
Bulk generation, high throughput
"sort_by": "tps", "sort_order": "max"
"sort_by": "requests", "sort_order": "min"
Picks the least loaded model
Load balancing
"sort_by": "requests", "sort_order": "min"
Complete Target Examples
Simple Array Targets
{
"targets": ["openai/gpt-4o", "anthropic/claude-4-sonnet"]
}
Provider Wildcard Targets
{
"targets": ["openai/*", "anthropic/*"]
}
Single Target
{
"targets": "openai/gpt-4o"
}
Advanced $any Pool with Filtering and Sorting
{
"targets": {
"$any": [
"openai/*",
"anthropic/claude-4-sonnet",
"mistral/mistral-large-latest"
],
"filter": {
"ttft": { "$lt": 1000 },
"provider.error_rate": { "$lt": 0.05 }
},
"sort_by": "price",
"sort_order": "min"
}
}
Filter Options
filter: { ttft: {...} }
Filter models by time-to-first-token
$lt
, $gt
, $lte
, $gte
, $eq
"ttft": { "$lt": 1000 }
filter: { price: {...} }
Filter models by cost
$lt
, $gt
, $lte
, $gte
, $eq
"price": { "$lt": 0.02 }
filter: { error_rate: {...} }
Filter models by reliability
$lt
, $gt
, $lte
, $gte
, $eq
"error_rate": { "$lt": 0.05 }
filter: { provider.error_rate: {...} }
Filter by provider reliability
$lt
, $gt
, $lte
, $gte
, $eq
"provider.error_rate": { "$lt": 0.02 }
filter: { model_tags: {...} }
Filter by model classification tags
$contains
, $in
"model_tags": { "$contains": "fast" }
Comparison Operators
$eq
Equals
"tier": { "$eq": "premium" }
Exact matching
$ne
Not equals
"tier": { "$ne": "free" }
Exclusion logic
$lt
Less than
"ttft": { "$lt": 1000 }
Performance thresholds
$lte
Less than or equal
"price": { "$lte": 0.02 }
Budget constraints
$gt
Greater than
"tps": { "$gt": 100 }
Minimum performance requirements
$gte
Greater than or equal
"reliability": { "$gte": 0.95 }
Quality standards
$in
In array
"region": { "$in": ["US", "EU"] }
Multiple value matching
$contains
Array contains value
"tags": { "$contains": "GDPR" }
Tag-based filtering
Important Notes:
Targets: Can be a simple array, single string, or
$any
object with filtering/sortingWildcards: Use
*
for provider-level routing (e.g.,"openai/*"
= all OpenAI models)Sorting: Use
"sort_by"
and"sort_order"
fields separatelyOperators: Always use
$
prefix for comparison operators ($eq
,$lt
, etc.)Values: Use lowercase for sort order (
"min"
,"max"
)
Last updated
Was this helpful?