Introducing Virtual MCP Servers
LogoLogo
GithubJoin SlackSignupBook a Demo
  • Documentation
  • Self Hosted
  • Integrations
  • Guides
  • Enterprise
  • Introduction to AI Gateway
  • Supported Models
  • Supported MCP Servers
  • Getting Started
    • Quick Start
    • Working with API
    • Working with Multiple Agents
    • Working with MCPs
    • Working with Headers
    • User Tracking
    • Using Parameters
  • Concepts
    • Thread
    • Trace
    • Run
    • Label
    • Message
    • Virtual Models
      • Routing with Virtual Model
    • Virtual MCP Servers
  • Features
    • Tracing
    • Routing
    • MCP Support
    • Publishing MCP Servers
    • Usage
    • Analytics
    • Guardrails
    • User Roles
    • Cost Control
  • Python SDK
    • Getting Started
  • API Reference
  • Postman Collection
Powered by GitBook
LogoLogo

Social

  • LinkedIn
  • X
  • Youtube
  • Github

Platform

  • Pricing
  • Documentation
  • Blog

Company

  • Home
  • About

Legal

  • Privacy Policy
  • Terms of Service

2025 LangDB. All rights reserved.

On this page

Was this helpful?

Export as PDF

Introduction to AI Gateway

Monitor, Govern and Secure your AI traffic.

NextQuick Start

Last updated 17 days ago

Was this helpful?

What is an AI Gateway?

An AI gateway is a middleware that acts as a unified access point to multiple LLMs, optimizing, securing, and managing AI traffic. It simplifies integration with different AI providers while enabling cost control, observability, and performance benchmarking. With an AI gateway, businesses can seamlessly switch between models, monitor usage, and optimize costs.

LangDB provides OpenAI compatible APIs to connect with multiple Large Language Models (LLMs) by just changing two lines of code.

Govern, Secure, and Optimize all of your AI Traffic with Cost Control, Optimisation and Full Observability.

What AI Gateway Offers Out of the Box

LangDB provides OpenAI-compatible APIs, enabling developers to connect with multiple LLMs by changing just two lines of code. With LangDB, you can:

  • Provide access to all major LLMs Ensure seamless integration with leading large language models to maximize flexibility and power.

  • No framework code required Enable plug-and-play functionality using any framework like Langchain, Vercel AI SDK, CrewAI, etc., for easy adoption.

  • Plug & Play Tracing & Cost Optimization Simplify implementation of tracing and cost optimization features, ensuring streamlined operations.

  • Automatic routing based on cost, quality, and other variables Dynamically route requests to the most suitable LLM based on predefined parameters.

  • Benchmark and provide insights Deliver insights into the best-performing models for specific tasks, such as coding or reasoning, to enhance decision-making.

Quick Start with LangDB

LangDB offers both managed and self hosted versions for organisations to manage AI traffic . Choose between the Hosted Gateway for ease of use or the Open-Source Gateway for full control.

Roadmap

  • Prompt Caching & Optimization (In Progress) Introduce caching mechanisms to optimize prompt usage and reduce redundant costs.

  • GuardRails (In Progress) Implement safeguards to enhance reliability and accuracy in AI outputs.

  • Leaderboard of models per category Create a comparative leaderboard to highlight model performance across categories.

  • Ready-to-use evaluations for non-data scientists Provide accessible evaluation tools for users without a data science background.

  • Readily fine-tunable data based on usage Offer pre-configured datasets tailored for fine-tuning, enabling customized improvements with ease.

A full featured and managed AI gateway that provides instant access to 250+ LLMs with enterprise ready features.

A self-hosted option for organizations that require complete control over their AI infrastructure.

Quick Start

Self Hosted

Diagram showing LangDB as a central AI gateway for applications, offering features like cost management, tracing, routing, prompt caching, security, and guardrails. LangDB connects to multiple AI providers, including OpenAI, Gemini, Anthropic, Bedrock, Mistral, Hugging Face, Cohere, and Llama.