> ## Documentation Index
> Fetch the complete documentation index at: https://docs.deeprails.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Comprehensive Safety

> One of the largest concerns with using LLMs in automation is that dangerous content could be output uncensored and exposed to thousands or millions of people before it's caught. The Comprehensive Safety metric evaluates whether a response is completely devoid of potentially dangerous statements of all categories.

export const DefinitionCard = ({children}) => {
  return <Card variant="secondary">
    <div style={{
    padding: '0.5rem',
    border: '5px solid var(--primary-light)',
    borderRadius: '0.5rem',
    fontSize: '1.3rem',
    lineHeight: '1.4',
    boxShadow: '0 0 10px 10px var(--primary-light)'
  }}>
        {children}
      </div>

</Card>;
};

export const Scale = ({low, mid, high, lowLabel = "Low", midLabel = "Mid", highLabel = "High", lowDescription, midDescription, highDescription, midColor = "yellow"}) => {
  return <div style={{
    display: 'flex',
    flexDirection: 'column',
    width: '100%'
  }}>
      <svg width="100%" height="30" style={{
    marginBottom: '8px'
  }}>
        <defs>
          <linearGradient id="redToGreen" x1="0%" y1="0%" x2="100%" y2="0%">
            <stop offset="0%" stopColor="red" />
            <stop offset="100%" stopColor="green" />
          </linearGradient>
        </defs>
        <rect width="100%" height="100%" fill="url(#redToGreen)" rx="4" ry="4" />
      </svg>

      <div style={{
    display: 'flex',
    justifyContent: 'space-between',
    width: '100%',
    marginBottom: '16px'
  }}>
        <p style={{
    margin: 0,
    fontSize: '12px'
  }}>{low}</p>
        {mid && <p style={{
    margin: 0,
    fontSize: '12px'
  }}>{mid}</p>}
        <p style={{
    margin: 0,
    fontSize: '12px'
  }}>{high}</p>
      </div>

      <div style={{
    display: 'flex',
    justifyContent: 'space-between',
    width: '100%'
  }}>
        <div style={{
    maxWidth: '40%'
  }}>
          <div style={{
    display: 'flex',
    alignItems: 'center',
    marginBottom: '4px'
  }}>
            <div style={{
    width: '12px',
    height: '12px',
    backgroundColor: 'red',
    borderRadius: '50%',
    marginRight: '8px'
  }}></div>
            <p style={{
    margin: 0,
    fontWeight: 'bold',
    fontSize: '14px'
  }}>{lowLabel}</p>
          </div>
          {lowDescription && <p style={{
    margin: 0,
    fontSize: '14px',
    color: '#666',
    maxWidth: '250px',
    lineHeight: '1.4'
  }}>{lowDescription}</p>}
        </div>
        {mid && <div style={{
    maxWidth: '40%',
    textAlign: 'center'
  }}>
            <div style={{
    display: 'flex',
    alignItems: 'center',
    justifyContent: 'center',
    marginBottom: '4px'
  }}>
              <div style={{
    width: '12px',
    height: '12px',
    backgroundColor: midColor,
    borderRadius: '50%',
    marginRight: '8px'
  }}></div>
              <p style={{
    margin: 0,
    fontWeight: 'bold',
    fontSize: '14px'
  }}>{midLabel}</p>
            </div>
            {midDescription && <p style={{
    margin: 0,
    fontSize: '14px',
    color: '#666',
    maxWidth: '250px',
    textAlign: 'center',
    lineHeight: '1.4'
  }}>{midDescription}</p>}
          </div>}


        <div style={{
    maxWidth: '40%',
    textAlign: 'right'
  }}>
          <div style={{
    display: 'flex',
    alignItems: 'center',
    justifyContent: 'flex-end',
    marginBottom: '4px'
  }}>
            <p style={{
    margin: 0,
    fontWeight: 'bold',
    fontSize: '14px'
  }}>{highLabel}</p>
            <div style={{
    width: '12px',
    height: '12px',
    backgroundColor: 'green',
    borderRadius: '50%',
    marginLeft: '8px'
  }}></div>
          </div>
          {highDescription && <p style={{
    margin: 0,
    fontSize: '14px',
    color: '#666',
    maxWidth: '250px',
    marginLeft: 'auto',
    lineHeight: '1.4'
  }}>{highDescription}</p>}
        </div>
      </div>
    </div>;
};

<DefinitionCard>
  <strong>Comprehensive Safety</strong> evaluates whether an AI-generated response contains unsafe, harmful, or prohibited content across a set of critical safety categories.
</DefinitionCard>

Comprehensive Safety is returned as a boolean. Detection of any safety violation in a response results in an automatic failure.

***

## Understanding Comprehensive Safety

Our Comprehensive Safety metric evaluates each model response across the following risk domains:

* **CBRN Misuse**: Chemical, Biological, Radiological, or Nuclear weapon-related content
* **Personally Identifiable Information (PII)**: Exposure of sensitive identity details like full names, SSNs, or contact information
* **Hate / Harassment**: Abusive, discriminatory, or hostile language targeting individuals or groups
* **Violence / Threats**: Physical threats, weapon use or creation instructions, or incitement to harm
* **Sexual Content**: Explicit or inappropriate sexual material, especially involving children
* **Self-Harm / Harmful Instructions**: Content that encourages self-injury or dangerous behaviors

## Evaluation Process

DeepRails performs a Multimodal Partitioned Evaluation of every model output to scan for safety violations. A few core pieces of logic ensure that the evaluation is as thorough and accurate as possible.

<Note>
  We intentionally describe evaluation logic at a high level to protect our IP. Exact segmentation, verification, and aggregation logic is proprietary.
</Note>

<Accordion title="Chunking">
  The model output is decomposed into up to 5 segments or 'chunks', usually one or more closely related sentences or clauses. Each chunk is evaluated for safety across each of the six risk domains.
</Accordion>

<Accordion title="Category Evaluation">
  Each chunk is assessed to determine if its content is related to any risk domains. For each domain identified, the evaluation model decides whether the content was discussed safely or not (a binary verdict).
</Accordion>

<Accordion title="All or Nothing Scoring">
  If any chunk is flagged for any category, then the entire response is failed. There is no partial credit assigned for Comprehensive Safety, due to the sensitive nature of the category.
</Accordion>

## Addressing Unsafe Responses

<Card>
  <div style={{display: 'flex', alignItems: 'center', gap: '0.5rem', marginBottom: '0.75rem'}}>
    <div style={{fontSize: '1.25rem', color: 'var(--primary-color)'}}>
      <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round">
        <path d="M12 20h9" />

        <path d="M16.5 3.5a2.121 2.121 0 0 1 3 3L7 19l-4 1 1-4L16.5 3.5z" />
      </svg>
    </div>

    <h3 style={{margin: 0, fontSize: '1.25rem', fontWeight: '600'}}>Improving Safety Outcomes</h3>
  </div>

  To minimize unsafe completions:

  <div style={{ marginTop: "1rem", paddingTop: "0.75rem", borderTop: "1px solid rgba(209, 213, 219, 0.33)" }}>
    <strong>Refine prompts and system instructions:</strong> Discourage content likely to result in triggering harmful categories.
  </div>

  <div style={{ marginTop: "0.75rem", paddingTop: "0.75rem", borderTop: "1px solid rgba(209, 213, 219, 0.33)" }}>
    <strong>Use category-level logging:</strong> Track which safety categories are most frequently violated and then guide your training or fine-tuning efforts to compensate.
  </div>

  <div style={{ marginTop: "0.75rem", paddingTop: "0.75rem", borderTop: "1px solid rgba(209, 213, 219, 0.33)" }}>
    <strong>Apply structured overrides:</strong> Use score-based logic to filter, flag, or escalate responses for human review in agentic workflows with chained LLMs.
  </div>
</Card>
