Respan focuses on LLM engineering, providing a central gateway and tracing layer for AI applications. It routes traffic to providers like OpenAI and Google Gemini, tracking tokens, costs, latency, and error rates. With gateway-based logging and an OpenTelemetry tracing SDK, engineers can inspect workflows from high-level tasks to individual model calls.
Platforms:
Unified LLM gateway routes requests through a single URL across various AI providers
Token, cost, and latency analytics display usage, cost, and error rates
Tracing SDK with decorators captures end-to-end traces for Python and JavaScript
Rich attribution metadata enables metrics slicing by user, project, experiment, or environment
Flexible logging modes offer proxy traffic or asynchronous logging options
