Provider Routing

Automatically route requests across multiple AI providers for optimal cost, latency, and accuracy. One model name, multiple providers.

Stop worrying about which provider to pick. With Provider Routing, you can call a model by name, and LangDB will automatically select the right provider for you.

Why Use Provider Routing?

• One Name, Many Providers – Call a model like deepseek-v3.1 and LangDB picks from DeepSeek official, Parasail, DeepInfra, Fireworks AI, and more.

• Optimize by Mode – Choose whether you want lowest cost, fastest latency, highest accuracy, or simply balanced routing.

---

Quick Start

{
  "model": "deepseek-v3.1",
  "messages": [
    {
      "role": "user",
      "content": "Explain reinforcement learning in simple terms."
    }
  ]
}

That’s it — LangDB will resolve deepseek-v3.1 across multiple providers, and by default use balanced mode.

---

Optimization Modes

When you specify only a model name, LangDB chooses the provider according to your selected mode.

Mode	What it does	Best for
balanced	Distributes requests across providers for optimal overall performance	General apps (default)
accuracy	Routes to the provider with the best benchmark score	Research, compliance
cost	Picks the cheapest provider by input/output token price	Support chatbots, FAQs
latency	Always selects the lowest latency provider	Real-time UIs, voice bots
throughput	Spreads requests across all providers to maximize concurrency	High-volume pipelines

---

Examples

Balanced (default)

{
  "model": "deepseek-v3.1",
  "messages": [{ "role": "user", "content": "Summarize this article." }]
}

LangDB chooses the provider dynamically, balancing cost, latency, and accuracy.

---

Cost Optimization

{
  "model": "deepseek-v3.1:cost",
  "messages": [{ "role": "user", "content": "Write a short FAQ response." }]
}

LangDB picks the cheapest provider for deepseek-v3.1 based on input/output token prices (e.g. Parasail, Fireworks AI, or DeepInfra if they’re lower than DeepSeek official).

---

Accuracy Optimization

{
  "model": "deepseek-v3.1:accuracy",
  "messages": [{ "role": "user", "content": "Solve this math word problem." }]
}

Routes to the provider with the highest benchmark score for deepseek-v3.1.

---

Latency Optimization

{
  "model": "deepseek-v3.1:latency",
  "messages": [{ "role": "user", "content": "Respond quickly for a live chat." }]
}

Always picks the provider with the fastest response times.

---

Throughput Optimization

{
  "model": "deepseek-v3.1:throughput",
  "messages": [{ "role": "user", "content": "Translate this dataset." }]
}

Distributes requests across all available providers for deepseek-v3.1 to maximize scale.

---

Explicit Provider Pinning

If you want full control, you can always specify the provider explicitly:

{
  "model": "parasail/deepseek-v3.1",
  "messages": [{ "role": "user", "content": "Generate a poem." }]
}

This bypasses provider routing and always uses the given provider.

---

Summary

• Use model without provider → LangDB does provider routing.

• Add :mode suffix → pick between balanced, accuracy, cost, latency, or throughput.

• Use provider/model → pin a specific provider directly.

Provider Routing makes it easy to scale across multiple vendors without rewriting your code.