Introducing Virtual MCP Servers
LogoLogo
GithubJoin SlackSignupBook a Demo
  • Documentation
  • Self Hosted
  • Integrations
  • Guides
  • Enterprise
  • Architecture Overview
  • Enterprise Licensing Options
  • Running Locally
    • ai-gateway.yaml
  • Tenant & User Provisioning
  • Deployment Options
    • Using Docker Compose
    • Deploying on AWS Cloud
    • Using Kubernetes (Beta)
    • Deploying on GCP (Beta)
  • Resources
    • Multi Tenancy
    • Configuring Data Retention
    • Clickhouse Queries
    • Working with Models
Powered by GitBook
LogoLogo

Social

  • LinkedIn
  • X
  • Youtube
  • Github

Platform

  • Pricing
  • Documentation
  • Blog

Company

  • Home
  • About

Legal

  • Privacy Policy
  • Terms of Service

2025 LangDB. All rights reserved.

On this page
  • Managing and Adding New Models
  • Management API to Publish New Models
  • Dynamic Request/Response Mapping (Coming Soon)

Was this helpful?

Export as PDF
  1. Resources

Working with Models

Manage public and private LLM models in LangDB with flexible APIs, custom parameter schemas, and dynamic request/response mapping.

PreviousClickhouse Queries

Last updated 15 hours ago

Was this helpful?

Managing and Adding New Models

The platform provides a flexible management API that allows you to publish and manage machine learning models at both the tenant and project levels. This enables organizations to control access and visibility of models, supporting both public and private use cases.

Model Types

  • Public Models:

    • Can be added without specifying a project ID.

    • Accessible to all users on this deployment.

    • In enterprise deployments, public models are added monthly.

    • Specific requests for models can be made talking to our support.

  • Private Models:

    • Require a project_id and a provider_id for a known provider with a pre-configured secret.

    • Access is restricted to the specified project and provider.

Model Parameters

When publishing a model, you can specify:

  • Request/Response Mapping:

    • By default, models are expected to be OpenAI-compatible.

    • You can also specify custom request/response processors using dynamic scripts (see 'Coming Soon' below).

  • Model Parameters Schema:

    • A JSON schema describing the parameters that can be sent with requests to the model.

Management API to Publish New Models

The management API allows you to register new models with the platform. Below is an example of how to use the API to publish a new model.

Sample cURL Request

curl -X POST https://api.xxx.langdb.ai/admin/models \
  -H "Authorization: Bearer <your_token>" \
  -H "X-Admin-Key: <admin_key>"\
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "my-model",
    "description" "Description of the LLM Model",
    "provider_id": "123e4567-e89b-12d3-a456-426614174000",
    "project_id": "<project_id>",
    "public": false,
    "request_response_mapping": "openai-compatible", // or custom script
    "model_type": "completions",
    "input_token_price": "0.00001",
    "output_token_price": "0.00003",
    "context_size": 128000,
    "capabilities": ["tools"],
    "input_types": ["text", "image"],
    "output_types": ["text", "image"],
    "tags": [],
    "owner_name": "openai",
    "priority": 0,
    "model_name_in_provider": "my-model-v1.2",
    "parameters": {
       "top_k":{
          "min":0,
          "step":1,
          "type":"int",
          "default":0,
          "required":false,
          "description":"Limits the token sampling to only the top K tokens. A value of 0 disables this setting, allowing the model to consider all tokens."
       },
       "top_p":{
          "max":1,
          "min":0,
          "step":0.05,
          "type":"float",
          "default":1,
          "required":false,
          "description":"An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. We generally recommend altering this or temperature but not both."
       },
    }
  }'

API properties

Field
Type
Description

model_name

String

The display name of the model

description

String

A detailed description of the model's capabilities and use cases

provider_info_id

UUID

The UUID of the provider that offers this model

project_id

UUID

Which project this model belongs to

public

Boolean

Whether the model is publicly discoverable or private

request_response_mapping

String

"openai-compatible" or a custom mapping script

model_type

String

The type of model (e.g., "completions", "image", "embedding")

owner_name

String

The name of the model's owner or creator

priority

i32

Priority level for the model in listings (higher numbers indicate higher priority)

input_token_price

Nullable float

Price per input token

output_token_price

Nullable float

Price per output token

context_size

Nullable u32

Maximum context window size in tokens

capabilities

String[]

List of model capabilities (e.g., "tools")

input_types

String[]

Supported input formats (e.g., "text", "image", "audio")

output_types

String[]

Supported output formats (e.g., "text", "image", "audio")

tags

String[]

Classification tags for the model

type_prices

Map<String, float>

JSON string containing prices for different usage types (used for image generation model pricing)

mp_price

Nullable float

Price by megapixel (used for image generation model pricing)

model_name_in_provider

String

The model's identifier in the provider's system

parameters

Map<String, Map<String, any>>

Additional configuration parameters as JSON

Usage of API

  • Replace <platform-url>, <your_token>, and <project_id> with your actual values.

  • Set public to true for public models (omit project_id and provider_id), or false for private models.

  • The parameters_schema field allows you to define the expected parameters for your model.

Dynamic Request/Response Mapping (Coming Soon)

The platform will soon support dynamic request/response mapping using custom scripts. This feature will allow you to define how requests are transformed before being sent to the model, and how responses are processed before being returned to the client. This will enable support for a wide variety of model APIs and custom workflows.

Stay tuned!

Checkout the full API Specification:

Model Listing on LangDB
Setting Parameters for a sample request on LangDB
LangDB Models listing page
Setting Parameters for a sample request on LangDB
#post-admin-models