> ## Documentation Index
> Fetch the complete documentation index at: https://docs.deeprails.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Instruction Adherence

> Prompts used in production workflows often are very complex and structured with dozens of validation rules. The Instruction Adherence metric assesses each of the input rules and evaluates whether the model response followed each consistently.

export const DefinitionCard = ({children}) => {
  return <Card variant="secondary">
    <div style={{
    padding: '0.5rem',
    border: '5px solid var(--primary-light)',
    borderRadius: '0.5rem',
    fontSize: '1.3rem',
    lineHeight: '1.4',
    boxShadow: '0 0 10px 10px var(--primary-light)'
  }}>
        {children}
      </div>

</Card>;
};

export const Scale = ({low, mid, high, lowLabel = "Low", midLabel = "Mid", highLabel = "High", lowDescription, midDescription, highDescription, midColor = "yellow"}) => {
  return <div style={{
    display: 'flex',
    flexDirection: 'column',
    width: '100%'
  }}>
      <svg width="100%" height="30" style={{
    marginBottom: '8px'
  }}>
        <defs>
          <linearGradient id="redToGreen" x1="0%" y1="0%" x2="100%" y2="0%">
            <stop offset="0%" stopColor="red" />
            <stop offset="100%" stopColor="green" />
          </linearGradient>
        </defs>
        <rect width="100%" height="100%" fill="url(#redToGreen)" rx="4" ry="4" />
      </svg>

      <div style={{
    display: 'flex',
    justifyContent: 'space-between',
    width: '100%',
    marginBottom: '16px'
  }}>
        <p style={{
    margin: 0,
    fontSize: '12px'
  }}>{low}</p>
        {mid && <p style={{
    margin: 0,
    fontSize: '12px'
  }}>{mid}</p>}
        <p style={{
    margin: 0,
    fontSize: '12px'
  }}>{high}</p>
      </div>

      <div style={{
    display: 'flex',
    justifyContent: 'space-between',
    width: '100%'
  }}>
        <div style={{
    maxWidth: '40%'
  }}>
          <div style={{
    display: 'flex',
    alignItems: 'center',
    marginBottom: '4px'
  }}>
            <div style={{
    width: '12px',
    height: '12px',
    backgroundColor: 'red',
    borderRadius: '50%',
    marginRight: '8px'
  }}></div>
            <p style={{
    margin: 0,
    fontWeight: 'bold',
    fontSize: '14px'
  }}>{lowLabel}</p>
          </div>
          {lowDescription && <p style={{
    margin: 0,
    fontSize: '14px',
    color: '#666',
    maxWidth: '250px',
    lineHeight: '1.4'
  }}>{lowDescription}</p>}
        </div>
        {mid && <div style={{
    maxWidth: '40%',
    textAlign: 'center'
  }}>
            <div style={{
    display: 'flex',
    alignItems: 'center',
    justifyContent: 'center',
    marginBottom: '4px'
  }}>
              <div style={{
    width: '12px',
    height: '12px',
    backgroundColor: midColor,
    borderRadius: '50%',
    marginRight: '8px'
  }}></div>
              <p style={{
    margin: 0,
    fontWeight: 'bold',
    fontSize: '14px'
  }}>{midLabel}</p>
            </div>
            {midDescription && <p style={{
    margin: 0,
    fontSize: '14px',
    color: '#666',
    maxWidth: '250px',
    textAlign: 'center',
    lineHeight: '1.4'
  }}>{midDescription}</p>}
          </div>}


        <div style={{
    maxWidth: '40%',
    textAlign: 'right'
  }}>
          <div style={{
    display: 'flex',
    alignItems: 'center',
    justifyContent: 'flex-end',
    marginBottom: '4px'
  }}>
            <p style={{
    margin: 0,
    fontWeight: 'bold',
    fontSize: '14px'
  }}>{highLabel}</p>
            <div style={{
    width: '12px',
    height: '12px',
    backgroundColor: 'green',
    borderRadius: '50%',
    marginLeft: '8px'
  }}></div>
          </div>
          {highDescription && <p style={{
    margin: 0,
    fontSize: '14px',
    color: '#666',
    maxWidth: '250px',
    marginLeft: 'auto',
    lineHeight: '1.4'
  }}>{highDescription}</p>}
        </div>
      </div>
    </div>;
};

<DefinitionCard>
  <strong>Instruction Adherence</strong> measures how closely a model response follows the instructions defined in the user and/or system prompts.
</DefinitionCard>

<Scale low="0" lowLabel="Low Adherence" high="1" highLabel="High Adherence" lowDescription="The response ignores or contradicts instructions" highDescription="The response follows all instructions precisely" />

***

## Understanding Instruction Adherence

<Card>
  <div style={{display: 'flex', alignItems: 'center', gap: '0.5rem', marginBottom: '0.75rem'}}>
    <div style={{fontSize: '1.25rem', color: 'var(--primary-color)'}}>
      <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round">
        <path d="M12 22c5.523 0 10-4.477 10-10S17.523 2 12 2 2 6.477 2 12s4.477 10 10 10z" />

        <path d="m9 12 2 2 4-4" />
      </svg>
    </div>

    <h3 style={{margin: 0, fontSize: '1.25rem', fontWeight: '600'}}>How Instruction Adherence Differs from Other Metrics</h3>
  </div>

  While it sounds similar at first, Instruction Adherence is distinctly different from other metrics:

  <div style={{ marginTop: "1rem", paddingTop: "0.75rem", borderTop: "1px solid rgba(209, 213, 219, 0.33)" }}>
    <strong>Instruction Adherence:</strong> Measures whether the response followed how it was supposed to answer—structure, tone, content constraints, formatting, etc.
  </div>

  <div style={{ marginTop: "0.75rem", paddingTop: "0.75rem", borderTop: "1px solid rgba(209, 213, 219, 0.33)" }}>
    <strong>Context Adherence:</strong> Measures whether the response reflects what was in the provided context (e.g., source documents).
  </div>

  <div style={{ marginTop: "0.75rem", paddingTop: "0.75rem", borderTop: "1px solid rgba(209, 213, 219, 0.33)" }}>
    <strong>Correctness:</strong> Measures whether the information in the response is factually accurate, regardless of whether it followed instructions or context.
  </div>
</Card>

## Evaluation Process

DeepRails performs a Multimodal Partitioned Evaluation of every model output to assess the extent to which it follows all prompt instructions. A few core pieces of logic ensure that the evaluation is as thorough and accurate as possible.

<Accordion title="Instruction Identification and Extraction">
  The model input is separated into explicit, atomic instructions. The most important claims are intelligently selected for evaluation to ensure the evaluation is completed in a timely manner.
</Accordion>

<Accordion title="Response Segmentation">
  The model output is decomposed into segments each relating to an identified instruction.
</Accordion>

<Accordion title="Confidence-based Adherence Judgment">
  Each segment analyzed and determined to either follow or not follow its instruction. For each binary verdict, a confidence rating is given as well.
</Accordion>

<Accordion title="Aggregate Scoring">
  All claim judgments are weighted by their confidence rating and consolidated into a final instruction adherence score between 0 and 1.
</Accordion>

## Addressing Low Instruction Adherence Scores

<Card>
  <div style={{display: 'flex', alignItems: 'center', gap: '0.5rem', marginBottom: '0.75rem'}}>
    <div style={{fontSize: '1.25rem', color: 'var(--primary-color)'}}>
      <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round">
        <path d="M12 20h9" />

        <path d="M16.5 3.5a2.121 2.121 0 0 1 3 3L7 19l-4 1 1-4L16.5 3.5z" />
      </svg>
    </div>

    <h3 style={{margin: 0, fontSize: '1.25rem', fontWeight: '600'}}>Improving Instruction Adherence</h3>
  </div>

  When models fail to follow instructions, the resulting output may be irrelevant or unusable in later production steps. To improve instruction-following:

  <div style={{ marginTop: "0.75rem", paddingTop: "0.75rem", borderTop: "1px solid rgba(209, 213, 219, 0.33)" }}>
    <strong>Refine prompts:</strong> Reword unclear or ambiguous instructions to be more direct, structured, and constraint-based.
  </div>

  <div style={{ marginTop: "0.75rem", paddingTop: "0.75rem", borderTop: "1px solid rgba(209, 213, 219, 0.33)" }}>
    <strong>Compare model variants:</strong> Some models are significantly more instruction-aligned than others. Use Adherence metrics to validate before selecting the model used in deployment.
  </div>
</Card>

## Best Practices

<CardGroup cols={2}>
  <Card title="Track Instruction Types" icon="list-tree">
    Categorize instructions (format, tone, scope, etc.) to make it harder for model's to miss them.
  </Card>

  <Card title="Design Robust Prompts" icon="wand-magic">
    Write instructions that are structured, unambiguous, and directive (e.g., “Respond only in bullet points” or “Return valid JSON”).
  </Card>
</CardGroup>

<Note>
  While the other metrics ensure the model gives all the right information, Instruction Adherence ensures that it gives it in the right way. This guardrail is essential for structured outputs, enterprise use cases, and task compliance.
</Note>
