Variables & Functions

Reference of router metrics, variables, filters, operators, and target formats.

Available Metrics

LangDB provides a rich set of real-time metrics for making dynamic, data-driven routing decisions. These metrics are aggregated at the provider level, giving you a live view of model performance.

Metric Name
Description
Example Value
Business Value

ttft

Time to First Token (ms)

450

Optimize for user-perceived speed

tps

Tokens Per Second (output_tokens/latency)

300

Measure model generation speed

error_rate

Fraction of failed requests

0.01

Route around unreliable models

latency

Average end-to-end response time (ms)

1100

Route based on overall performance

requests

Total number of requests processed

1500

Monitor traffic and usage patterns

cost

Estimated request cost (USD)

0.02

Track and control spend in real time

Available Variables

Variables provide contextual information from the incoming request and user metadata. Unlike metrics, they are not performance indicators but are essential for conditional logic.

User Information

Variable Path
Description
Example Value
Business Use

extra.user.id

User identifier from request

"u-12345"

Custom user identification

extra.user.name

User name from request

"john.doe"

Personalization

extra.user.email

User email from request

User-based routing

extra.user.tier

User tier from request

"premium"

SLA-based routing

extra.user.tiers

Array of user tiers from request

["premium", "beta"]

Multi-tier access control

extra.user.*

Any custom field in extra.user

Various

Custom business logic

Request Metadata

Variable Path
Description
Example Value
Business Use

metadata.ip

IP address of the requester

"203.0.113.42"

Geo-fencing, fraud detection

metadata.region

Geographical region (based on IP)

"EU"

Data residency, compliance

metadata.user_agent

Client application

"Google ADK/ CrewAI"

Agentic library detection

metadata.user_id

LangDB user ID

"langdb-u-12345"

Internal user identification

metadata.group_id

LangDB cost group ID

"grp-456"

Cost group identification

metadata.group_name

LangDB cost group name

"development"

Cost group-based routing

metadata.tags.{tag}

LangDB user tags

"premium"

User classification

Pre-Request Interceptor Results

Variable Path
Description
Example Value
Business Use

pre_request.{interceptor_name}.passed

Whether interceptor check passed

true

Conditional routing based on checks

pre_request.rate_limiter.passed

Rate limit check result

true

Enforce usage quotas

pre_request.semantic_guardrail.passed

Content policy check result

false

Block policy violations

pre_request.{guardrail}.result.topic

Detected topic from guardrail

"billing"

Topic-based model routing

pre_request.{guardrail}.result.*

Any custom result from interceptor

Various

Custom business logic

Provider Performance Metrics (for Filtering)

Variable Path
Description
Example Value
Business Use

provider.error_rate

Provider's current error rate

0.02

Avoid unreliable providers

model_capabilities

Supported features (vision, code)

["vision"]

Feature-based routing

model_tags

Model classification tags

["fast"]

Tag-based filtering

Target Configuration

Configuration Type
Description
Example Usage
When to Use

Simple List

Direct array of model names

["openai/gpt-4o", "anthropic/claude-4-sonnet"]

Fixed model selection

Provider Wildcards

All models from specific provider

["openai/*", "anthropic/*"]

Provider-level routing

Single Target

Direct model specification

"openai/gpt-4o"

Fixed routing to one model

$any Pool

Pool of models with selection logic

{ "$any": [...], "sort_by": "price" }

Dynamic model selection

Model Target Formats

Format
Description
Example
Use Case

Exact Model Name

Specific model identifier

"openai/gpt-4o"

Direct routing

Provider Wildcard

All models from provider

"openai/*"

Provider-level failover

Sorting Options

Sort Configuration
What It Does
When to Use
Example JSON Usage

"sort_by": "price", "sort_order": "min"

Picks the cheapest model

Cost control, bulk/low-priority tasks

"sort_by": "price", "sort_order": "min"

"sort_by": "price", "sort_order": "max"

Picks the most expensive model

Premium users, highest quality needed

"sort_by": "price", "sort_order": "max"

"sort_by": "ttft", "sort_order": "min"

Picks the fastest model (time-to-first-token)

VIP, real-time, or user-facing tasks

"sort_by": "ttft", "sort_order": "min"

"sort_by": "latency", "sort_order": "min"

Picks the lowest latency model

Performance-critical applications

"sort_by": "latency", "sort_order": "min"

"sort_by": "error_rate", "sort_order": "min"

Picks the most reliable model

Mission-critical or regulated workflows

"sort_by": "error_rate", "sort_order": "min"

"sort_by": "tps", "sort_order": "max"

Picks the fastest token generation model

Bulk generation, high throughput

"sort_by": "tps", "sort_order": "max"

"sort_by": "requests", "sort_order": "min"

Picks the least loaded model

Load balancing

"sort_by": "requests", "sort_order": "min"

Complete Target Examples

Simple Array Targets

{
  "targets": ["openai/gpt-4o", "anthropic/claude-4-sonnet"]
}

Provider Wildcard Targets

{
  "targets": ["openai/*", "anthropic/*"]
}

Single Target

{
  "targets": "openai/gpt-4o"
}

Advanced $any Pool with Filtering and Sorting

{
  "targets": {
    "$any": [
      "openai/*",
      "anthropic/claude-4-sonnet",
      "mistral/mistral-large-latest"
    ],
    "filter": {
      "ttft": { "$lt": 1000 },
      "provider.error_rate": { "$lt": 0.05 }
    },
    "sort_by": "price",
    "sort_order": "min"
  }
}

Filter Options

Filter Type
Description
Operators Supported
Example Usage

filter: { ttft: {...} }

Filter models by time-to-first-token

$lt, $gt, $lte, $gte, $eq

"ttft": { "$lt": 1000 }

filter: { price: {...} }

Filter models by cost

$lt, $gt, $lte, $gte, $eq

"price": { "$lt": 0.02 }

filter: { error_rate: {...} }

Filter models by reliability

$lt, $gt, $lte, $gte, $eq

"error_rate": { "$lt": 0.05 }

filter: { provider.error_rate: {...} }

Filter by provider reliability

$lt, $gt, $lte, $gte, $eq

"provider.error_rate": { "$lt": 0.02 }

filter: { model_tags: {...} }

Filter by model classification tags

$contains, $in

"model_tags": { "$contains": "fast" }

Comparison Operators

Operator
Description
Example Usage
Use Case

$eq

Equals

"tier": { "$eq": "premium" }

Exact matching

$ne

Not equals

"tier": { "$ne": "free" }

Exclusion logic

$lt

Less than

"ttft": { "$lt": 1000 }

Performance thresholds

$lte

Less than or equal

"price": { "$lte": 0.02 }

Budget constraints

$gt

Greater than

"tps": { "$gt": 100 }

Minimum performance requirements

$gte

Greater than or equal

"reliability": { "$gte": 0.95 }

Quality standards

$in

In array

"region": { "$in": ["US", "EU"] }

Multiple value matching

$contains

Array contains value

"tags": { "$contains": "GDPR" }

Tag-based filtering

Important Notes:

  • Targets: Can be a simple array, single string, or $any object with filtering/sorting

  • Wildcards: Use * for provider-level routing (e.g., "openai/*" = all OpenAI models)

  • Sorting: Use "sort_by" and "sort_order" fields separately

  • Operators: Always use $ prefix for comparison operators ($eq, $lt, etc.)

  • Values: Use lowercase for sort order ("min", "max")

Last updated

Was this helpful?