Virtual Models

Create, save, and reuse LLM configurations with Virtual Models in LangDB AI Gateway to streamline workflows and ensure consistent behavior.

LangDB’s Virtual Models let you save, share, and reuse model configurations—combining prompts, parameters, tools, and routing logic into a single named unit. This simplifies workflows and ensures consistent behavior across your apps, agents, and API calls.

Once saved, these configurations can be quickly accessed and reused across multiple applications.

Why do you need Virtual Models

Virtual models in LangDB are more than just model aliases. They are fully configurable AI agents that:

• Let you define system/user messages upfront

• Support routing logic to dynamically choose between models

• Include MCP integrations and guardrails

• Are callable from UI playground, API, and LangChain/OpenAI SDKs

Use virtual models to manage:

• Prompt versioning and reuse

• Consistent testing across different models

• Precision tuning with per-model parameters

• Seamless integration of tools and control logic

• Routing using strategies like fallback, percentage-based, latency-based, optimized, and script-based selection

Setting Up Virtual Model

1. Go to the Models

2. Click on Create Virtual Model.

3. Set prompt messages — define system and user messages to guide model behavior

4. Set variables (optional) — useful if your prompts require dynamic values

5. Select router type

   • None: Use a single model only

   • Fallback, Random, Cost,Percentage, Latency, Optimized: Configure smart routing across targets. Checkout all Routing Strategies.

6. Add one or more targets

   • Each target defines a model, mcp servers, guardrails, system-user messages, response format and its parameters (e.g. temperature, max_tokens, top_p, penalties)

7. Select MCP Servers — connect tools like LangDB Search, Code Execution, or others

8. Add guardrails (optional) — for validation, transformation, or filtering logic

9. Set response format — choose between text, json_object, or json_schema

10. Give your virtual model a name and Save.

Your virtual model now appears in the Models section of your project, ready to be used anywhere a model is accepted.

Updating and Versioning

You can edit virtual models anytime. LangDB supports formal versioning via the @version syntax:

• langdb/my-model@latest or langdb/my-model → resolves to the latest version

• langdb/my-model@v1 or langdb/my-model@1 → resolves to version 1

This allows you to safely test new versions, roll back to older ones, or maintain multiple stable variants of a model in parallel.

Using Your Virtual Model

Once saved, your virtual model is fully available across all LangDB interfaces:

• Chat Playground: Select it from the model dropdown and test interactively.

• OpenAI-Compatible SDKs: Works seamlessly with OpenAI clients by changing only the model name.

• LangChain / CrewAI / other frameworks: Call it just like any base model by using model="langdb/my-model@latest" or a specific version like @v1.

This makes virtual models a portable, modular building block across all parts of your AI stack.