Example: Routing with Interceptors and Compliance

Example showing rate limiting, semantic guardrails, GDPR routing, and error handling.

This example showcases a sophisticated routing configuration that uses pre-request interceptors to enforce usage quotas and guardrails, while handling region-specific compliance and prioritizing performance for premium users.

Goals:

1. Enforce a daily rate limit on all users to prevent abuse.

2. Check all requests for policy violations using a semantic guardrail.

3. Provide high-performance models for premium users in the EU, but only if they are reliable.

4. Ensure GDPR compliance by using a specialized model for requests with that requirement.

5. Provide a clear error message to users who have exceeded their quota.

Complete Chat Completion Request with Interceptors and Compliance:

{
  "model": "router/dynamic",
  "messages": [
    {
      "role": "system",
      "content": "You are a financial advisory assistant that provides investment recommendations and portfolio analysis for premium clients in the European market."
    },
    {
      "role": "user",
      "content": "I need urgent advice on my cryptocurrency portfolio diversification strategy. Given the current market volatility, should I reallocate my €2M investment across different digital assets? Please provide detailed analysis including tax implications under GDPR compliance requirements."
    }
  ],
  "stream": true,
  "extra": {
    "user": {
      "tier": "premium"
    }
  },
  "router": {
    "type": "conditional",
    "pre_request": [
      {
        "name": "rate_limiter",
        "type": "interceptor",
        "limit": 1000,
        "period": "day",
        "target": "user_id"
      },
      {
        "name": "semantic_guardrail",
        "type": "guardrail"
      }
    ],
    "routes": [
      {
        "name": "premium_eu_high_performance",
        "conditions": {
          "all": [
            {
              "extra.user.tier": {
                "$eq": "premium"
              }
            },
            {
              "metadata.region": {
                "$eq": "EU"
              }
            },
            {
              "pre_request.rate_limiter.passed": {
                "$eq": true
              }
            }
          ]
        },
        "targets": {
          "$any": [
            "anthropic/claude-4-opus",
            "openai/gpt-o3",
            "gemini/gemini-2.5-pro"
          ],
          "filter": {
            "error_rate": {
              "$lt": 0.02
            }
          },
          "sort_by": "ttft",
          "sort_order": "min"
        }
      },
      {
        "name": "gdpr_compliance_fallback",
        "conditions": {
          "metadata.tags.gdpr": {
            "$eq": "true"
          }
        },
        "targets": "eu-specialist/gdpr-compliant-model"
      },
      {
        "name": "semantic_guardrail_violation_block",
        "conditions": {
          "pre_request.semantic_guardrail.passed": {
            "$eq": false
          }
        },
        "message_mapper": {
          "modifier": "block",
          "content": "Your request contains content that violates our usage policies. Please review your message and try again with appropriate content."
        }
      },
      {
        "name": "rate_limit_exceeded_block",
        "conditions": {
          "pre_request.rate_limiter.passed": {
            "$eq": false
          }
        },
        "message_mapper": {
          "modifier": "block",
          "content": "You have exceeded your daily quota of 1,000 requests. Please try again tomorrow or upgrade to enterprise tier for higher limits."
        }
      },
      {
        "name": "fallback_route",
        "conditions": {
          "all": [
            {
              "pre_request.rate_limiter.passed": {
                "$eq": true
              }
            },
            {
              "pre_request.semantic_guardrail.passed": {
                "$eq": true
              }
            }
          ]
        },
        "targets": "openai/gpt-4o-mini"
      }
    ],
    "max_retries": 2
  }
}

Configuration Breakdown:

• Request Structure:

  • Uses "model": "router/dynamic" to enable dynamic routing

  • User information and compliance requirements are passed via the extra.user object

  • Router configuration includes pre_request interceptors and conditional routing

• Pre-Request Interceptors: Before routing evaluation, two interceptors run:

  • rate_limiter: Enforces 1,000 requests per day limit per user. Results are accessible via pre_request.rate_limiter.passed

  • semantic_guardrail: Scans content for policy violations. Results are accessible via pre_request.semantic_guardrail.passed (custom guardrail implementation required)

• Route 1: premium_eu_high_performance

  • Conditions: Requires premium tier (extra.user.tier), EU region (metadata.region), rate limit not exceeded, and low provider error rate

  • Targets: Routes to high-performance models with error rate filtering and minimum time-to-first-token sorting

• Route 2: gdpr_compliance_fallback

  • Conditions: Checks for GDPR compliance requirement via metadata.tags.gdpr (automatically set by LangDB from x-tags header)

  • Targets: Routes to specialized EU GDPR-compliant model regardless of user tier

• Route 3: semantic_guardrail_violation_block

  • Conditions: Triggers when semantic guardrail fails (pre_request.semantic_guardrail.passed = false)

  • Action: Uses message_mapper to block request and return policy violation error message

• Route 4: rate_limit_exceeded_block

  • Conditions: Triggers when rate limiter fails (pre_request.rate_limiter.passed = false)

  • Action: Uses message_mapper to block request and return informative error message with upgrade suggestion

• Route 5: fallback_route

  • Conditions: Only applies to requests that pass both rate limiting and semantic guardrail checks

  • Targets: Routes to openai/gpt-4o-mini as reliable fallback for valid requests that don't match other specific conditions

Key Features Demonstrated:

• Pre-Request Processing: Rate limiting and content guardrails before routing

• Compliance Routing: Automatic GDPR-compliant model selection

• Error Handling: Custom error messages for quota exceeded scenarios

• Performance Optimization: Provider health checks and fast response prioritization

• Regional Awareness: EU-specific routing based on metadata.region (automatically set by LangDB based on user location)