DeepRails Monitor gives you full observability into your production AI workflows.
LLM-powered applications can behave in unexpected ways. For mission-critical generative generative AI applications, visibility into how models perform is essential for ensuring performance, reliability, and positive user experience. DeepRails Monitor gives you full observability into your generative AI application in production. It lets you track token usage, performance trends, user interactions, and quality signals using Guardrail metrics—all in one place. When issues arise, the Monitor dashboard helps you drill into failures, regressions, and edge cases so you can diagnose root causes and deploy fixes with speed and confidence.

​
Core Features

  • Real-Time Monitoring
    Continuously observe LLM outputs and performance in production. Stay ahead of issues with real-time insights and activity traces.
  • Guardrail Metrics
    Apply research-backed Guardrail metrics such as correctness, completeness, adherance, and safety directly to live traffic.
  • Insights and Analysis
    Detect critical errors, behavioral shifts, and anomalies across your application. Drill down into affected runs for root cause analysis and uncover underlying issues.
  • Cost & Usage Visibility
    Monitor token usage, frequency, latency, and cost across workloads to optimize for both quality and efficiency.

Krino Console: Real-Time Monitoring Dashboard

​
The Workflow

1

Set Up Your Monitor and Metrics

Configure your Monitor for a specific workflow and select the relevant Guardrail metrics to track quality and performance over time.
2

Log Your Production Traffic

Send your live completions, prompts, and model metadata to DeepRails via API. All traffic is automatically tracked and evaluated.
3

Analyze Results

Use the Krino Console to visualize trends, detect performance regressions, identify failures, and review Guardrail scoring across time windows.
4

Debug, Re-Test, and Iterate

Isolate failing flows or prompts. Run targeted evaluations to validate fixes, or use Defend to enforce real-time guardrails.

​
Getting Started

Quickstart