Skip to main content

​
Create an API Key

  1. In your organization’s DeepRails API Console, go to API Keys.
  2. Click Create key, name it, then copy the key.
  3. (Optional) Save it as the DEEPRAILS_API_KEY environment variable.
API Keys – placeholder

Create and manage API keys in the API Console.

​
Install the SDK

  • Python
  • TypeScript / Node
  • Ruby
  • Go
pip install deeprails

​
Create a Monitor

Before you can send events, you need to create a monitor. A monitor is a container for tracking production events and their evaluations. In this example, the evaluations in the monitor will leverage a file as context, which will need to be uploaded first.
Tip: You can also create a monitor via the DeepRails API Console.
  • Python
  • TypeScript / Node
  • Ruby
  • Go
from deeprails import DeepRails

# Initialize (env var DEEPRAILS_API_KEY is recommended)
client = DeepRails(api_key="YOUR_API_KEY")

try:

    file_id = ""
    with open("example.txt", "r") as file:
        # Read the entire content of the file
        content = file.read()
        file_response = client.files.upload(content)

    file_id = file_response.file_id

    # Create a monitor
    monitor = client.monitor.create(
        name="Production Chat Assistant Monitor",
        description="Monitoring our production chatbot responses",
        guardrail_metrics=[
          "completeness",
          "correctness"
        ],
        web_search=True,
        file_search=[
          file_id
        ]
    )
    
    print(f"Monitor created:\n{monitor}")

except Exception as e:
    print(f"Error: {e}")
Required Parameters
FieldTypeDescription
namestringThe human-readable name of the monitor
guardrail_metricsstring[]A list of one or more metrics that events associated with this monitor will be evaluated on
Optional Parameters
FieldTypeDescription
descriptionstringA description of the monitor and/or the associated production use case
web_searchboolWhether or not web search is added as an extended capability for this monitor’s evaluations. Defaults to false
file_searchstring[]A list of file IDs to be used for file search in this monitor’s evaluations. If nothing is passed, file search will not be used

​
Send Your First Monitor Event

Use the SDK to log a production event (input + output). The SDK automatically triggers an evaluation of the guardrail metrics assigned to the monitor and links the result to the event. Retrieve the details of the event until it gives a completed status; then, you can view the outcome of the evaluation.
  • Python
  • TypeScript / Node
  • Ruby
  • Go
from deeprails import DeepRails
import time

# Initialize (env var DEEPRAILS_API_KEY is recommended)
client = DeepRails(api_key="YOUR_API_KEY")

# Create a monitor event (get the monitor_id from Console → Monitors)
created = client.monitor.submit_event(
    monitor_id="mon_xxxxxxxxxxxxxxxxxxxx",
    model_input={"user_prompt": "Summarize the Paris Agreement in 3 bullets."},
    model_output="• International treaty on climate change...",
    run_mode="economy",
)

print(created)

time.sleep(5)

status = ""
while status is not "completed":
  time.sleep(1)
  event = client.monitor.retrieve_event(event_id=created.event_id)
  status = event.status

print(event.evaluation_result)
Required Parameters
FieldTypeDescription
monitor_idstringThe ID of the monitor to receive the event (find it in Console → Monitor → Manage Monitors).
model_inputobjectMust include at least system_prompt or user_prompt.
model_outputstringThe LLM output to be evaluated and recorded with the event.
Optional Parameters
FieldTypeDescription
run_modestringRun mode for the workflow event that determines which models are used to evaluate the event. Available run modes include precision_plus, precision, smart, and economy. Defaults to smart.

​
Retrieve Monitor Data

You can retrieve a monitor’s details via API including a list of recent events and stats on associated evaluation progress.
  • Python
  • TypeScript / Node
  • Ruby
  • Go
from deeprails import DeepRails

client = DeepRails(api_key="YOUR_API_KEY")

try:
    # Get monitor details
    monitor = client.monitor.retrieve(monitor_id="mon_xxxxxxxxxxxxxxxxxxxx")
    print(monitor)
except Exception as e:
    print(f"Error: {e}")

​
Check Monitor Analytics via the API Console

  1. Open DeepRails API Console → Monitor → Data.
  2. Filter by model, time range, or search by monitor_id to find events.
  3. Open any event to see the linked evaluation scores and rationales.
Monitor data – placeholder

Browse real-time monitor events, filters, and linked evaluation details.

​
Next Steps

Learn About Guardrails

Explore the metrics behind evaluations—correctness, safety, completeness, and more.

Monitor Overview

Learn core Monitor concepts and how it fits into your flow.