Introducing Virtual MCP Servers
LogoLogo
GithubJoin SlackSignupBook a Demo
  • Documentation
  • Self Hosted
  • Integrations
  • Guides
  • Enterprise
  • Introduction to AI Gateway
  • Supported Models
  • Supported MCP Servers
  • Getting Started
    • Quick Start
    • Working with API
    • Working with Multiple Agents
    • Working with MCPs
    • Working with Headers
    • User Tracking
    • Using Parameters
  • Concepts
    • Thread
    • Trace
    • Run
    • Label
    • Message
    • Virtual Models
      • Routing with Virtual Model
    • Virtual MCP Servers
  • Features
    • Tracing
    • Routing
    • MCP Support
    • Publishing MCP Servers
    • Usage
    • Analytics
    • Guardrails
    • User Roles
    • Cost Control
  • Python SDK
    • Getting Started
  • API Reference
  • Postman Collection
Powered by GitBook
LogoLogo

Social

  • LinkedIn
  • X
  • Youtube
  • Github

Platform

  • Pricing
  • Documentation
  • Blog

Company

  • Home
  • About

Legal

  • Privacy Policy
  • Terms of Service

2025 LangDB. All rights reserved.

On this page
  • Installation
  • Initialize LangDB Client
  • Making a Chat Completion Request
  • Retrieve Messages from a Thread
  • Get Thread Usage
  • Get Analytics
  • Evaluate Multiple Threads
  • List Available Models
  • Usage in Evaluation

Was this helpful?

Export as PDF
  1. Python SDK

Getting Started

Use LangDB’s Python SDK to generate completions, monitor API usage, retrieve analytics, and evaluate LLM workflows efficiently.

LangDB simplifies working with multiple Large Language Models (LLMs) through a single API. It excels at analytics, usage monitoring, and evaluation, giving developers insights into model performance, usage stats, and costs. This guide covers installation, setup, and key functionalities.

Installation

To install the LangDB Python client, run:

pip install pylangdb

Initialize LangDB Client

Initialize the client with your API key and project ID:

from pylangdb import LangDb

client = LangDb(
    api_key="your_api_key",
    project_id="your_project_id"
)

Making a Chat Completion Request

You can generate a response using the completion method:

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Say hello!"}
]

response = client.completion(
    model="gemini-1.5-pro-latest",
    messages=messages,
    temperature=0.7,
    max_tokens=100
)

print(response["content"])  # Output AI response

Retrieve Messages from a Thread

You can fetch messages from a specific thread using its thread_id:

thread_id = response["thread_id"]
messages = client.get_messages(thread_id)

for message in messages:
    print(f"Type: {message.type}")
    print(f"Content: {message.content}")
    if message.tool_calls:
        for tool_call in message.tool_calls:
            print(f"Tool: {tool_call.function.name}")

Get Thread Usage

Retrieve cost and token usage details for a thread:

usage = client.get_usage(thread_id)
print(f"Total Cost: ${usage.total_cost:.4f}")
print(f"Input Tokens: {usage.total_input_tokens}")
print(f"Output Tokens: {usage.total_output_tokens}")

Get Analytics

You can retrieve analytics for specific model tags:

analytics_data = client.get_analytics(tags="gpt-4,gemini")
print(analytics_data)

Alternatively, you can convert analytics data into a Pandas DataFrame for easier analysis:

import pandas as pd

df = client.get_analytics_dataframe(tags="gpt-4,gemini")
print(df.head())

Evaluate Multiple Threads

To generate an evaluation DataFrame containing message and cost information for multiple threads:

df = client.create_evaluation_df(thread_ids=["thread1", "thread2"])
print(df)

List Available Models

To list all models supported by LangDB:

models = client.list_models()
print(models)

Usage in Evaluation

LangDB provides built-in evaluation capabilities, allowing developers to assess model performance, response accuracy, and cost efficiency. By analyzing messages, token usage, and analytics data, teams can fine-tune their models for better results.

PreviousCost ControlNextAPI Reference

Last updated 17 days ago

Was this helpful?