Skip to main content
POST
/
defend
from deeprails import Deeprails

DEEPRAILS_API_KEY = "YOUR_API_KEY"

client = Deeprails(
    api_key=DEEPRAILS_API_KEY, 
)

workflow_response = client.defend.create_workflow(
  improvement_action="fixit",
  metrics={
      "completeness": 0.0,
      "instruction_adherence": 0.0,
  },
  name="Test Workflow",
  type="automatic",
  automatic_tolerance="high"
)
print(workflow_response.workflow_id)
{
  "workflow_id": "<string>",
  "name": "<string>",
  "description": "<string>",
  "improvement_action": "regenerate",
  "status": "archived",
  "created_at": "2023-11-07T05:31:56Z",
  "modified_at": "2023-11-07T05:31:56Z",
  "success_rate": 123,
  "max_retries": 123
}
The request body must include a name for the workflow, a workflow type, a metrics object specifying which guardrail metrics to evaluate against, and an improvement_action. If workflow type is automatic, then automatic_tolerance must be specified to inform threshold selection.

The type of the workflow determines how hallucination thresholds are configured:
- automatic - Uses internal threshold logic based on tolerance level. The automatic_tolerance field must be specified as low, medium, or high to indicate how strict the thresholds should be. When automatic is selected, the float values in the metrics dictionary are ignored, as the automatic_tolerance parameter determines the threshold instead.
- custom - Allows the user to set custom thresholds (0.0-1.0) for each metric.

Improvement actions determine the remediation step for events that have one or more metric evaluations fail below the threshold.
- regenerate - Re-runs the user’s prompt with minor variance
- fixit - Directly addresses shortcomings using the guardrail failure rationale(s)
- null - No improvement is attempted by DeepRails

When you create a workflow, you’ll receive a workflow ID. Use this ID to submit new events for evaluation and remediation for the defend workflow.

Authorizations

​
Authorization
string
header
required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Body

application/json
​
name
string
required

Name of the workflow.

​
type
enum<string>
required

Type of thresholds to use for the workflow, either automatic or custom. Automatic thresholds are assigned internally after the user specifies a qualitative tolerance for the metrics, whereas custom metrics allow the user to set the threshold for each metric as a floating point number between 0.0 and 1.0.

Available options:
automatic,
custom
​
metrics
object
required

Mapping of guardrail metrics to floating point threshold values. If the workflow type is automatic, only the metric names are used (automatic_tolerance determines thresholds). Possible metrics are correctness, completeness, instruction_adherence, context_adherence, ground_truth_adherence, or comprehensive_safety.

​
improvement_action
enum<string> | null
required

The action used to improve outputs that fail one or guardrail metrics for the workflow events. May be regenerate, fixit, or null which represents “do nothing”. Regenerate runs the user's input prompt with minor induced variance. Fixit attempts to directly address the shortcomings of the output using the guardrail failure rationale. Do nothing does not attempt any improvement.

Available options:
regenerate,
fixit
​
description
string

Description for the workflow.

​
automatic_tolerance
enum<string>

Hallucination tolerance for automatic workflows; may be low, medium, or high. Ignored if type is custom.

Available options:
low,
medium,
high
​
max_retries
integer
default:10

Max. number of improvement action retries until a given event passes the guardrails. Defaults to 10.

Response

Workflow created successfully

​
workflow_id
string
required

A unique workflow ID.

​
name
string
required

Name of the workflow.

​
description
string

Description for the workflow.

​
improvement_action
enum<string> | null

The action used to improve outputs that fail one or more guardrail metrics for the workflow events. May be regenerate, fixit, or null which represents “do nothing”. Regenerate runs the user's input prompt with minor induced variance. Fixit attempts to directly address the shortcomings of the output using the guardrail failure rationale. Do nothing does not attempt any improvement.

Available options:
regenerate,
fixit
​
status
enum<string>

Status of the selected workflow. May be archived or active. Archived workflows will not accept events.

Available options:
archived,
active
​
created_at
string<date-time>

The time the workflow was created in UTC.

​
modified_at
string<date-time>

The most recent time the workflow was modified in UTC.

​
success_rate
number

Rate of events associated with this workflow that passed evaluation.

​
max_retries
integer

Max. number of improvement action retries until a given event passes the guardrails.

⌘I