Rate Limiter Routing

Apply per-user quotas and route based on rate-limit results.

Enforce usage quotas and route requests accordingly. Block users who exceed their limits while providing clear error messages.

Use Case

• Prevent API abuse

• Manage costs with usage quotas

• Provide clear feedback on rate limits

Configuration

{
  "model": "router/dynamic",
  "router": {
    "type": "conditional",
    "pre_request": [
      {
        "name": "daily_user_limit",
        "type": "rate_limiter",
        "limit": 2,
        "period": "hour",
        "entity": "user_id",
        "target": "requests"
      }
    ],
    "routes": [
      {
        "conditions": {
          "all": [
            {
              "pre_request.daily_user_limit.allowed": {
                "$eq": true
              }
            }
          ]
        },
        "name": "main_requests",
        "targets": {
          "$any": [
            "anthropic/*"
          ],
          "sort_by": "price",
          "sort_order": "min"
        }
      },
      {
        "name": "other_requests",
        "targets": {
          "$any": [
            "openai/gpt-4.1-nano"
          ],
          "sort_by": "price",
          "sort_order": "min"
        }
      }
    ]
  }
}

How It Works

1. Rate Limiter: Checks if user has exceeded 2 requests per hour

2. Main Requests: Routes allowed requests to any Anthropic model, sorted by lowest price

3. Other Requests: Routes remaining requests to GPT-4.1-nano for cost optimization

Variables Used

• pre_request.daily_user_limit.allowed: Result of rate limit check (true if within limit)

Customization

• Adjust limits (daily, weekly, monthly)

• Different limits per user tier

• Custom error messages

• Graceful degradation instead of blocking