> ## Documentation Index
> Fetch the complete documentation index at: https://docs.deeprails.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Context Adherence

> One of the most powerful aspects of generative AI is models' abilities to adjust to a given context. As such, monitoring the model's adherence is the most critical metric for many use cases. Note, however, the evaluation will not work effectively if DeepRails does not detect enough context in the prompt. Only use Context Adherence when a notable set of context is passed along with the prompt.

export const DefinitionCard = ({children}) => {
  return <Card variant="secondary">
    <div style={{
    padding: '0.5rem',
    border: '5px solid var(--primary-light)',
    borderRadius: '0.5rem',
    fontSize: '1.3rem',
    lineHeight: '1.4',
    boxShadow: '0 0 10px 10px var(--primary-light)'
  }}>
        {children}
      </div>

</Card>;
};

export const Scale = ({low, mid, high, lowLabel = "Low", midLabel = "Mid", highLabel = "High", lowDescription, midDescription, highDescription, midColor = "yellow"}) => {
  return <div style={{
    display: 'flex',
    flexDirection: 'column',
    width: '100%'
  }}>
      <svg width="100%" height="30" style={{
    marginBottom: '8px'
  }}>
        <defs>
          <linearGradient id="redToGreen" x1="0%" y1="0%" x2="100%" y2="0%">
            <stop offset="0%" stopColor="red" />
            <stop offset="100%" stopColor="green" />
          </linearGradient>
        </defs>
        <rect width="100%" height="100%" fill="url(#redToGreen)" rx="4" ry="4" />
      </svg>

      <div style={{
    display: 'flex',
    justifyContent: 'space-between',
    width: '100%',
    marginBottom: '16px'
  }}>
        <p style={{
    margin: 0,
    fontSize: '12px'
  }}>{low}</p>
        {mid && <p style={{
    margin: 0,
    fontSize: '12px'
  }}>{mid}</p>}
        <p style={{
    margin: 0,
    fontSize: '12px'
  }}>{high}</p>
      </div>

      <div style={{
    display: 'flex',
    justifyContent: 'space-between',
    width: '100%'
  }}>
        <div style={{
    maxWidth: '40%'
  }}>
          <div style={{
    display: 'flex',
    alignItems: 'center',
    marginBottom: '4px'
  }}>
            <div style={{
    width: '12px',
    height: '12px',
    backgroundColor: 'red',
    borderRadius: '50%',
    marginRight: '8px'
  }}></div>
            <p style={{
    margin: 0,
    fontWeight: 'bold',
    fontSize: '14px'
  }}>{lowLabel}</p>
          </div>
          {lowDescription && <p style={{
    margin: 0,
    fontSize: '14px',
    color: '#666',
    maxWidth: '250px',
    lineHeight: '1.4'
  }}>{lowDescription}</p>}
        </div>
        {mid && <div style={{
    maxWidth: '40%',
    textAlign: 'center'
  }}>
            <div style={{
    display: 'flex',
    alignItems: 'center',
    justifyContent: 'center',
    marginBottom: '4px'
  }}>
              <div style={{
    width: '12px',
    height: '12px',
    backgroundColor: midColor,
    borderRadius: '50%',
    marginRight: '8px'
  }}></div>
              <p style={{
    margin: 0,
    fontWeight: 'bold',
    fontSize: '14px'
  }}>{midLabel}</p>
            </div>
            {midDescription && <p style={{
    margin: 0,
    fontSize: '14px',
    color: '#666',
    maxWidth: '250px',
    textAlign: 'center',
    lineHeight: '1.4'
  }}>{midDescription}</p>}
          </div>}


        <div style={{
    maxWidth: '40%',
    textAlign: 'right'
  }}>
          <div style={{
    display: 'flex',
    alignItems: 'center',
    justifyContent: 'flex-end',
    marginBottom: '4px'
  }}>
            <p style={{
    margin: 0,
    fontWeight: 'bold',
    fontSize: '14px'
  }}>{highLabel}</p>
            <div style={{
    width: '12px',
    height: '12px',
    backgroundColor: 'green',
    borderRadius: '50%',
    marginLeft: '8px'
  }}></div>
          </div>
          {highDescription && <p style={{
    margin: 0,
    fontSize: '14px',
    color: '#666',
    maxWidth: '250px',
    marginLeft: 'auto',
    lineHeight: '1.4'
  }}>{highDescription}</p>}
        </div>
      </div>
    </div>;
};

<DefinitionCard>
  <strong>Context Adherence</strong> measures how closely an AI response aligns with the information in the context provided in or with the user prompt. **This metric is only appropriate for prompts that include a significant context window.**
</DefinitionCard>

<Scale low="0" lowLabel="Low Adherence" high="1" highLabel="High Adherence" lowDescription="Response includes unsupported or out-of-context statements" highDescription="Response only includes information verifiable from the provided context" />

***

## Understanding Context Adherence

<Card>
  <div style={{display: 'flex', alignItems: 'center', gap: '0.5rem', marginBottom: '0.75rem'}}>
    <div style={{fontSize: '1.25rem', color: 'var(--primary-color)'}}>
      <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round">
        <path d="M12 22c5.523 0 10-4.477 10-10S17.523 2 12 2 2 6.477 2 12s4.477 10 10 10z" />

        <path d="m9 12 2 2 4-4" />
      </svg>
    </div>

    <h3 style={{margin: 0, fontSize: '1.25rem', fontWeight: '600'}}>Context Adherence vs. Other Metrics</h3>
  </div>

  Context Adherence is critical in tasks that require strict fidelity to provided documents or sources.

  <div style={{ marginTop: "1rem", paddingTop: "0.75rem", borderTop: "1px solid rgba(209, 213, 219, 0.33)" }}>
    <strong>Context Adherence:</strong> Measures whether the response reflects the information available in the provided context (e.g., retrieved documents, input references).
  </div>

  <div style={{ marginTop: "0.75rem", paddingTop: "0.75rem", borderTop: "1px solid rgba(209, 213, 219, 0.33)" }}>
    <strong>Correctness:</strong> Measures whether the response is factually accurate based on external truth—regardless of the provided context.
  </div>

  <div style={{ marginTop: "0.75rem", paddingTop: "0.75rem", borderTop: "1px solid rgba(209, 213, 219, 0.33)" }}>
    <strong>Instruction Adherence:</strong> Measures whether the model followed *how* it was supposed to answer, based on explicit instructions (e.g., tone, format).
  </div>
</Card>

***

## Evaluation Process

DeepRails performs a Multimodal Partitioned Evaluation of every model output to assess whether each claim is grounded in the context of the input to the model. Since adherence metrics involve more analysis of the model input than metrics like Correctness, the evaluation flow is a bit more complex.

```mermaid theme={null}
flowchart LR
  A["Context Identification"]
  B["Claim Identification"]
  C1["Insufficient Context"]
  C2["Claim/Context Matching"]
  D["Evaluation and Y/N Verdict Assignment"]
  E["Confidence Assignment"]
  F["Score Aggregation"]

  A --> B
  B --> C1
  B --> C2
  C2 --> D
  D --> E
  E --> F
```

<Note>
  Context Adherence evaluations will terminate prematurely if insufficient context is detected, and the output will be given a 100% score as a default.
</Note>

***

## Addressing Low Context Adherence Scores

<Card>
  <div style={{display: 'flex', alignItems: 'center', gap: '0.5rem', marginBottom: '0.75rem'}}>
    <div style={{fontSize: '1.25rem', color: 'var(--primary-color)'}}>
      <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round">
        <path d="M12 20h9" />

        <path d="M16.5 3.5a2.121 2.121 0 0 1 3 3L7 19l-4 1 1-4L16.5 3.5z" />
      </svg>
    </div>

    <h3 style={{margin: 0, fontSize: '1.25rem', fontWeight: '600'}}>Improving Context Adherence</h3>
  </div>

  To reduce out-of-context generations and improve grounding quality:

  <div style={{ marginTop: "1rem", paddingTop: "0.75rem", borderTop: "1px solid rgba(209, 213, 219, 0.33)" }}>
    <strong>Use clear and complete context:</strong> You should ensure that all facts needed for an ideal response are included in the context window.
  </div>

  <div style={{ marginTop: "0.75rem", paddingTop: "0.75rem", borderTop: "1px solid rgba(209, 213, 219, 0.33)" }}>
    <strong>Instruct against extrapolation:</strong> You should direct the model to avoid speculation or deviation in all prompts that use a context window.
  </div>

  <div style={{ marginTop: "0.75rem", paddingTop: "0.75rem", borderTop: "1px solid rgba(209, 213, 219, 0.33)" }}>
    <strong>Audit across prompt types:</strong> Compare how different tasks (e.g., summarization, QA) influence context drift, and use different models, prompts, and/or context windows for each.
  </div>
</Card>

***

<Note>
  Context Adherence is critical for grounded generation tasks like RAG, search, summarization, and citation. However, it should not be used in context-light applications.
</Note>
