Interceptors & Guardrails

How to use pre-request interceptors and guardrails in the router with examples.

Interceptors and Guardrails

Interceptors are custom logic that can run before or after a request is routed, allowing you to enrich, validate, or transform requests and responses. Guardrails are a common type of interceptor used to enforce policies.

Type
Purpose
Business Value
Example Configuration

pre_request

Analyze or enrich request

Classify topic, check for risk, personalize

rate_limiter, semantic_guardrail, toxicity_guardrail

post_request

Analyze or modify response

Moderate output, add fallback, redact sensitive info

fallback_response, content_moderator

Pre-Request Interceptor Configuration

Pre-request interceptors are defined in the routing.pre_request array and run before routing evaluation:

{
  "routing": {
    "pre_request": [
      { 
        "name": "rate_limiter", 
        "type": "interceptor", 
        "limit": 1000, 
        "period": "day", 
        "target": "user_id" 
      },
      { 
        "name": "semantic_guardrail", 
        "type": "guardrail" 
      }
    ]
  }
}

Interceptor Result Variables

When an interceptor runs, it injects its results into the routing context using the pre_request.{interceptor_name} namespace:

Result Variable Path
Description
Example Value
Business Use

pre_request.{interceptor_name}.passed

Whether the interceptor check passed

true

Conditional routing based on checks

pre_request.rate_limiter.passed

Rate limit check result

false

Enforce usage quotas and prevent abuse

pre_request.semantic_guardrail.passed

Content policy check result

true

Block policy violations

pre_request.toxicity_guardrail.passed

Toxicity check result

false

Block harmful content

pre_request.{guardrail}.result.topic

Detected topic from semantic analysis

"billing"

Route to topic-specialized models

pre_request.{guardrail}.result.score

Confidence or toxicity score

0.8

Threshold-based routing decisions

pre_request.{guardrail}.result.*

Any custom result from interceptor

Various

Custom business logic

Common Interceptor Types

Interceptor Type
Purpose
Configuration Fields
Result Fields

rate_limiter

Enforce request quotas

limit, period, target

passed

semantic_guardrail

Content classification and filtering

Custom guardrail configuration

passed, result.topic, result.*

toxicity_guardrail

Detect harmful content

Custom guardrail configuration

passed, result.score

custom_interceptor

Business-specific logic

Varies by implementation

passed, result.*

Usage in Routing Conditions

Use interceptor results in your routing conditions:

{
  "conditions": {
    "all": [
      { "pre_request.rate_limiter.passed": { "$eq": true } },
      { "pre_request.semantic_guardrail.passed": { "$eq": true } },
      { "pre_request.semantic_guardrail.result.topic": { "$eq": "support" } }
    ]
  }
}

Error Handling with Message Mappers

Block requests that fail interceptor checks:

{
  "name": "rate_limit_block",
  "conditions": {
    "pre_request.rate_limiter.passed": { "$eq": false }
  },
  "message_mapper": {
    "modifier": "block",
    "content": "Rate limit exceeded. Please try again later."
  }
}

Note on Custom Guardrails: Guardrails like semantic_guardrail and toxicity_guardrail are powerful examples of custom guardrails. Check out the Guardrails section for implementation details and configuration options.

Last updated

Was this helpful?