Variables & Functions

Available Metrics

LangDB provides a rich set of real-time metrics for making dynamic, data-driven routing decisions. These metrics are aggregated at the provider level, giving you a live view of model performance.

Metric Name	Description	Example Value	Business Value
requests	Total number of requests processed	1500	Monitor traffic and usage patterns
input_tokens	Number of tokens in the request prompt	500	Analyze prompt complexity and cost
output_tokens	Number of tokens in the model response	1200	Track response length and cost
total_tokens	Sum of input and output tokens	1700	Get a complete picture of token usage
latency	Average end-to-end response time (ms)	1100	Route based on overall performance
ttft	Time to First Token (ms)	450	Optimize for user-perceived speed
llm_usage	Aggregated cost of LLM usage	2.54	Track and control spend in real-time
tps	Tokens Per Second (output_tokens/latency)	300	Measure model generation speed
error_rate	Fraction of failed requests	0.01	Route around unreliable models

Available Variables

Variables provide contextual information from the incoming request and user metadata. Unlike metrics, they are not performance indicators but are essential for conditional logic.

Request Information

Variable	Description	Example Value	Business Use
ip	IP address of the requester	"203.0.113.42"	Geo-fencing, fraud detection
region	Geographical region of request	"EU"	Data residency, compliance
user_agent	Client application	"Google ADK/ CrewAI"	Agentic Library used

User Information

Variable	Description	Example Value	Business Use
user_id	Unique user identifier	"u-12345"	Auditing, per-user quotas
user_tier	User segment (e.g., premium, free)	"premium"	Personalization, rate limiting, SLAs
group	User group or segment	"beta_testers"	Feature rollout, A/B testing
region	Geographical region of request	"EU"	Data residency, compliance

Provider Metadata

Variable	Description	Example Value	Business Use
model_family	Model family or provider	"openai/gpt-4"	Brand preference, compliance
capabilities	Supported features (vision, code)	["vision", "code"]	Feature-based routing
compliance_tags	Regulatory/compliance attributes	["GDPR", "HIPAA"]	Regulatory routing

Optimisation Functions

Function	What It Does	When to Use	Example JSON Usage
sort: { price: MIN }	Picks the cheapest model	Cost control, bulk/low-priority tasks	"sort": { "price": "MIN" }
sort: { ttft_ms: MIN }	Picks the fastest model (latency)	VIP, real-time, or user-facing tasks	"sort": { "ttft": "MIN" }
sort: { error_rate: MIN }	Picks the most reliable model	Mission-critical or regulated workflows	"sort": { "error_rate": "MIN" }
sort: { token: MIN }	(Token-based) Picks model with lowest token cost	Optimize for token spend (current)	"sort": { "token": "MIN" }

Note: Currently, token-based optimization is available and recommended for controlling LLM costs.