Introduction to AI Gateway

LangDB provides OpenAI compatible APIs to connect with multiple Large Language Models (LLMs) with a single line of code change.

Just change base_url and extra_headers to get full tracing and observability.

You can use any of the supported models.

Copy# Import OpenAI library
import openai
from openai import OpenAI

client = OpenAI(
    base_url="https://api.us-east-1.langdb.ai"  # LangDB API base URL,
    api_key=api_key,  # Replace with your LangDB token
)
messages = [
    {
        "role": "system",
        "content": "You are a financial assistant. Help the user with their financial queries regarding companies.", 
    },
    {
        "role": "user",
        "content": "What are the earnings of Apple in 2022?",
    },
]
# Make the API call to LangDB's Completions API
response = client.chat.completions.create(
    model="gpt-4o",  # Use the model
    messages=messages,  # Define the interaction
    temperature=0.7,  # Control the creativity of the response
    max_tokens=300,  # Limit the length of the response
    top_p=1.0,  # Use nucleus sampling
    extra_headers={"x-project-id": "xxxxx"} # LangDB Project ID
)

Features:

• Cost Management: Track and control LLM usage to optimize spending.

• Dynamic Routing: Automatically direct requests to the most appropriate LLM based on requirements.

• Scalability: Simplifies scaling across projects and environments without added complexity.

Built for developers, the AI Gateway focuses on providing a practical and streamlined experience for integrating LLMs into your workflows.