Using Router

LangDB AI Gateway provides an intuitive user interface (UI) to configure and manage routing strategies. Through the UI, users can set up model routing dynamically, ensuring requests are directed efficiently based on cost, speed, and performance.

Accessing the Router Configuration

1

Open your Project on LangDB

2

Navigate to Router Section

3

Create a New Router

Let's create a router for routing based Let's create a TTFT-Based Router, which optimizes request handling by directing traffic to the model with the lowest TTFT. This ensures the fastest initial response, making it ideal for real-time applications like chatbots.

UI

1. Select type: Optimized

2. Select metric: TTFT

3. Select Target Models and Parameters

JSON

You can define router using JSON. There are examples in the Routing Documentation.

For example to define the same TTFT-Optimized router as in the UI.