You can monitor API usage with key insights.
After integrating LangDB into your project, the Analytics Dashboard becomes your central hub for understanding usage.
Metrics
LangDB’s Analytics Dashboard is segmented into several key panels:
1. Cost:
-
Tracks your total cost consumption across all integrated models.
-
Enables you to compare costs by provider/model/tags, helping you identify the most cost-effective options for your use cases.
2. Time:
-
Displays the average duration of requests in milliseconds.
-
Useful for benchmarking response times and optimizing performance for latency-sensitive applications.
3. Number of Requests:
-
Shows the total number of API calls made.
-
Helps you analyze usage patterns and allocate resources effectively.
4. Average Time to First Token (TTFT)
-
Indicates the average time taken to receive the first token from the API response.
-
This metric is critical for understanding initial latency.
5. Tokens Per Second (TPS)
-
Measures the throughput of token generation.
-
High TPS is indicative of efficient processing.
6. Time Per Output Token (TPOT)
-
Tracks the average time spent per output token.
-
Helps in identifying and troubleshooting bottlenecks in model output.